hadoop2.2.0 搭建初体验

殇帝刘玢你

下载 hadoop-2.2.0.tar.gz

tar-xzvf hadoop-2.2.0.tar.gz
cdhadoop-2.2.0
vietc/hadoop/hadoop-env.sh
修改：
export JAVA_HOME=/usr/local/jdk1.7.0_15
vietc/hadoop/slaves
修改localhost为hostmaster
vietc/hadoop/core-site.xml
增加：
<property>
<name>fs.defaultFS</name>
<value>hdfs://hostmaster:9000</value>
</property>
cpetc/hadoop/mapred-site.xml.template etc/hadoop/mapred-site.xml
vietc/hadoop/mapred-site.xml
增加：
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.cluster.local.dir</name>
<value>/home/ysc/mapreduce/local</value>
</property>
vietc/hadoop/yarn-site.xml
增加：
<property>
<name>yarn.resourcemanager.hostname</name>
<value>host001</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
vietc/hadoop/hdfs-site.xml
增加：
<property>
<name>dfs.name.dir</name>
<value>你的目录/dfs/filesystem/name</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>你的目录/dfs/filesystem/data</value>
</property>
<property>
<name>dfs.replication</name>
<value>3</value> <!--为分布式改为1-->

</property>
sudo vi /etc/profile
增加：
export HADOOP_PREFIX=/home/ysc/hadoop-2.2.0
export HADOOP_COMMON_HOME=/home/ysc/hadoop-2.2.0
export HADOOP_MAPRED_HOME=/home/ysc/hadoop-2.2.0
export HADOOP_CONF_DIR=/home/ysc/hadoop-2.2.0/etc/hadoop
export HADOOP_HDFS_HOME=/home/ysc/hadoop-2.2.0
export HADOOP_YARN_HOME=/home/ysc/hadoop-2.2.0
source/etc/profile
格式化：
bin/hdfs namenode -format
启动dfs：
sbin/start-dfs.sh
启动yarn：
sbin/start-yarn.sh
运行wordcount:
echo "Hello this is first hadoop2.2.0" > text1.txt
echo "start stop start stop" > text2.txt
bin/hadoop fs -mkdir input
bin/hadoop fs -put text1.txt input
bin/hadoop fs -put text2.txt input
bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar wordcount input output
访问管理页面
http://hostmaster:8088
http://hostmaster:50070
停止dfs：
sbin/stop-dfs.sh
停止yarn：
sbin/stop-yarn.sh