Fading Coder

One Final Commit for the Last Sprint

Home > Tech > Content

Enterprise Deployment of HDP 3.3.1 Orchestrated via Ambari on CentOS 7

Tech Jul 11 1

Infrastructure Specifications

Validate that target nodes satisfy the baseline resource thresholds before initiating provisioning:

Node Identifier Hardware Profile Network Endpoint Primary Function
mgmt-controller 8 vCPU / 16 GiB RAM / 300 GB SSD 10.50.20.10 Ambari Server & Monitoring
compute-node-alpha 8 vCPU / 16 GiB RAM / 300 GB SSD 10.50.20.11 Cluster Agent
compute-node-beta 8 vCPU / 16 GiB RAM / 300 GB SSD 10.50.20.12 Cluster Agent

Host Preconfiguration

Establish zero-trust SSH authentication across the fleet and apply kernel parameters tuned for distributed workloads.

# Generate RSA keypair without passphrase
ssh-keygen -t rsa -N '' -f ~/.ssh/id_rsa -q

# Broadcast authorized keys to target machines
for host_addr in 10.50.20.10 10.50.20.11 10.50.20.12; do
    ssh-copy-id "$host_addr"
done

Assign distinct FQDNs and enforce static DNS resolution:

hostnamectl set-hostname mgmt-controller
hostnamectl set-hostname compute-node-alpha
hostnamectl set-hostname compute-node-beta

cat >> /etc/hosts << HOSTS
10.50.20.10 mgmt-controller
10.50.20.11 compute-node-alpha
10.50.20.12 compute-node-beta
HOSTS

Neutralize memory fragmentation risks and rsetrict swapping behavior:

echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled
echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag
echo 'vm.swappiness=10' >> /etc/sysctl.conf
sysctl -p

Database Foundation

Provision a dedicated relational store to persist orchestration metadata.

# Register community repository and install server package
rpm -Uvh http://repo.mysql.com/mysql57-community-release-el7.rpm
yum install -y mysql-community-server

# Patch systemd pre-execution script to permit unencrypted bootstrapping
sed -i 's/--initialize /--initialize-insecure /' /usr/bin/mysqld_pre_systemd

# Write optimized configuration profile
cat > /etc/my.cnf << CNF
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
bind-address=0.0.0.0
port=3306
character-set-server=utf8mb4
collation-server=utf8mb4_unicode_ci
max_connections=1200
innodb_buffer_pool_size=2G
innodb_log_file_size=256M
binlog_format=row
log_error=/var/log/mysqld.log
lower_case_table_names=1
skip-name-resolve

[client]
default-character-set=utf8mb4
socket=/var/lib/mysql/mysql.sock
CNF

systemctl enable mysqld && systemctl start mysqld

# Secure the administrative account
mysql -u root -e "ALTER USER 'root'@'localhost' IDENTIFIED BY 'ProdDb!Str0ng'; FLUSH PRIVILEGES;"

Instantiate the schema and allocate scoped credentials for the manager service:

mysql -u root -p'ProdDb!Str0ng' << DDL
CREATE DATABASE ambari_meta CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
GRANT ALL PRIVILEGES ON ambari_meta.* TO 'ops_ambari'@'%' IDENTIFIED BY 'OpsGr4nt!Pass';
FLUSH PRIVILEGES;
USE ambari_meta;
SOURCE /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql;
DDL

Artifact Repository Provisioning

Deploy a lightweight HTTP service to distribute offline packages efficiently.

yum install -y httpd createrepo
rm -f /etc/httpd/conf.d/welcome.conf

# Tune server directives for high-throughput delivery
sed -i 's/^ServerTokens .*/ServerTokens Prod/' /etc/httpd/conf/httpd.conf
sed -i 's/^AllowOverride None/AllowOverride All/' /etc/httpd/conf/httpd.conf

systemctl enable httpd && systemctl start httpd

# Establish directory layout and unpack distribution bundles
mkdir -p /var/www/html/{ambari,hdp,hdp-utils}
tar xf ambari-2.7.6.0-22-centos7.tar.gz -C /var/www/html/ambari
tar xf HDP-3.3.1.0-001-centos7.tar.gz -C /var/www/html/hdp
tar xf HDP-UTILS-1.1.0.22-centos7.tar.gz -C /var/www/html/hdp-utils

chown -R apache:apache /var/www/html/
createrepo /var/www/html/hdp/

Management Server Bootstrapping

Register local repositories, deploy prerequisites, and initialize the central coordinator.

cat > /etc/yum.repos.d/ambari.repo << YUMCFG
[ambari]
name=Apache Ambari Repository
baseurl=http://10.50.20.10/ambari/2.7.6.0-22/
enabled=1
gpgcheck=0
YUMCFG

# Install JRE across every cluster member
rpm -ivh jdk-8u292-linux-x64.rpm --nodeps --force

# Map JDBC driver to system libraries
mkdir -p /usr/share/java
cp mysql-connector-java-5.1.49.jar /usr/share/java/mysql-connector-java.jar

# Deploy core orchestrator
yum install -y ambari-server

# Execute silent configuration routine
ambari-server setup --skip-database-verification --non-interactive

# Inject connection parameters programmatically
ambari-server setup-db \
  --database=mysql \
  --dbhost=10.50.20.10 \
  --dbport=3306 \
  --dbname=ambari_meta \
  --dbuser=ops_ambari \
  --dbpassword=OpsGr4nt!Pass \
  --non-interactive

ambari-server start

Cluster Onboarding & Interface Access

Invoke the browser-based console to map services across physical assets.

# Management endpoint becomes active upon successful daemon launch
Target: http://10.50.20.10:8080
Credentials: admin / admin

The provisioning wizard exposes available technology stacks. Select the HDP 3.3.1 lineage, categorize hosts by availability zone or rack, verify network reachability, and execute the automated topology ganeration workflow.

Tags: HDPAmbari

Related Articles

Understanding Strong and Weak References in Java

Strong References Strong reference are the most prevalent type of object referencing in Java. When an object has a strong reference pointing to it, the garbage collector will not reclaim its memory. F...

Comprehensive Guide to SSTI Explained with Payload Bypass Techniques

Introduction Server-Side Template Injection (SSTI) is a vulnerability in web applications where user input is improper handled within the template engine and executed on the server. This exploit can r...

Implement Image Upload Functionality for Django Integrated TinyMCE Editor

Django’s Admin panel is highly user-friendly, and pairing it with TinyMCE, an effective rich text editor, simplifies content management significantly. Combining the two is particular useful for bloggi...

Leave a Comment

Anonymous

◎Feel free to join the discussion and share your thoughts.