Use the following instructions to manually add a slave node:
On each new slave node, configure the remote repository as described here.
On each new slave node, install HDFS as described here.
On each new slave node, install compression libraries as described here.
On each new slave node, create the DataNode and YARN NodeManager local directories as described in section 4.3 on this page.
Copy the Hadoop configurations to the new slave nodes and set appropriate permissions.
Option I: Copy Hadoop config files from an existing slave node.
On an existing slave node, make a copy of the current configurations:
tar zcvf hadoop_conf.tgz /etc/hadoop/conf
Copy this file to each of the new nodes:
rm -rf /etc/hadoop/conf cd / tar zxvf $location_of_copied_conf_tar_file/hadoop_conf.tgz chmod -R 755 /etc/hadoop/conf
Option II: Manually set up the Hadoop configuration files as described here.
On each of the new slave nodes, start the DataNode:
su -l hdfs -c "/usr/lib/hadoop/sbin/hadoop-daemon.sh --config /etc/hadoop/conf start datanode"
On each of the new slave nodes, start the NodeManager:
su - yarn -c "export HADOOP_LIBEXEC_DIR=/usr/lib/hadoop/libexec && /usr/lib/hadoop-yarn/sbin/yarn-daemon.sh --config /etc/hadoop/conf start nodemanager"
Optional - If you use a HDFS or YARN/ResourceManager
.includefile in your cluster, add the new slave nodes to the.includefile, then run the applicablerefreshNodescommand.To add new DataNodes to the
dfs.includefile:On the NameNode host machine, edit the
/etc/hadoop/conf/dfs.includefile and add the list of the new slave node hostnames (separated by newline character).![[Note]](../common/images/admon/note.png)
Note If no
dfs.includefile is specified, all DataNodes are considered to be included in the cluster (unless excluded in thedfs.excludefile). Thedfs.hostsanddfs.hosts.exludeproperties inhdfs-site.xmlare used to specify thedfs.includeanddfs.excludefiles.On the NameNode host machine, execute the following command:
su -l hdfs -c "hdfs dfsadmin -refreshNodes"
To add new NodeManagers to the
yarn.includefile:On the ResourceManager host machine, edit the
/etc/hadoop/conf/yarn.includefile and add the list of the slave node hostnames (separated by newline character).![[Note]](../common/images/admon/note.png)
Note If no
yarn.includefile is specified, all NodeManagers are considered to be included in the cluster (unless excluded in theyarn.excludefile). Theyarn.resourcemanager.nodes.include-pathandyarn.resourcemanager.nodes.exclude-pathproperties inyarn-site.xmlare used to specify theyarn.includeandyarn.excludefiles.On the ResourceManager host machine, execute the following command:
su -l yarn -c "yarn rmadmin -refreshNodes"

