安装hadoop-2.3.0-cdh5.1.2
记录安装步骤，网上有点乱安装过程简单，主要是前面步骤做好就行
参考地址 http://blog.csdn.net/qyf_5445/article/details/42679857
包下载地址
jdk
http://www.oracle.com/technetwork/java/javase/downloads/java-archive-downloads-javase7-521261.htmljdk1.7.0_79
hadoop
http://archive.cloudera.com/cdh5/cdh/5/hadoop-2.3.0-cdh5.1.2
主机名 ip 角色
master192.168.5.30 NameNode ResourceManager
slave1 192.168.5.31 DateNode NodeManager
slave2 192.168.5.32 DateNode NodeManager
以下操作3个节点都需要做
1.cat /etc/hosts
192.168.5.30 master
192.168.5.31 slave1
192.168.5.32 slave2
2.
useradd hadoop
passwd hadoop
3.ssh互信,su hadoop(这步在master做即可)
ssh-keygen -t rsa
ssh-copy-id -i>
ssh-copy-id -i> 4.
mkdir -p /data/hadoop
chown -R hadoop.hadoop /data/hadoop/
5.java环境
cat /etc/profile
export JAVA_HOME=/usr/java/jdk1.7.0_79
export JAVA_BIN=$JAVA_HOME/bin
export PATH=$PATH:$JAVA_HOME/bin
export> export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH
export HADOOP_HOME=/data/hadoop
export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$JAVA_HOME/bin:$PATH
------------------------------------------------------
以下在master上操作
6.修改/etc/hadoop/hadoop-env.sh中设JAVA_HOME。
应当使用绝对路径。
export JAVA_HOME=$JAVA_HOME //错误，不能这么改
export JAVA_HOME=/usr/java/jdk1.7.0_79 //正确，应该这么改
7.
tar xvf hadoop-2.3.0-cdh5.1.2.tar.gz -C /data/hadoop
确认目录结构是/data/hadoop/etc/hadoop
8.master上/data/hadoop/etc/hadoop
vi slaves
slave1
slave2
vi masters
master
9.修改如下几个文件，并在中间添加以下相应内容
vi core-site.xml
fs.defaultFS
hdfs://master:9000
io.file.buffer.size
131072
hadoop.tmp.dir
file:/data/hadoop/tmp
vi hdfs-site.xml
dfs.namenode.name.dir
file:/data/hadoop/dfs/name
dfs.namenode.data.dir
file:/data/hadoop/dfs/data
dfs.replication
2
vi yarn-site.xml
yarn.resourcemanager.address
master:8032
yarn.resourcemanager.scheduler.address
master:8030
yarn.resourcemanager.resource-tracker.address
master:8031
yarn.resourcemanager.admin.address
master:8033
yarn.resourcemanager.webapp.address
master:8088
yarn.nodemanager.aux-services
mapreduce_shuffle
yarn.nodemanager.aux-services.mapreduce.shuffle.class
org.apache.hadoop.mapred.ShuffleHandler
vi mapred-site.xml
mapreduce.framework.name
yarn
mapreduce.jobhistory.address
master:10020
mapreduce.jobhistory.webapp.address
master:19888
10.复制hadoop到slave1,slave2
su hadoop
scp -r /data/hadoop/* hadoop@slave1:/data/hadoop/
scp -r /data/hadoop/* hadoop@slave2:/data/hadoop/
11.su hadoop,hadoop namenode -format初始化只需要做一次
12.su hadoop,start-dfs.sh and start-yarn.sh
提示logging to /data/hadoop/logs/hadoop-hadoop-namenode-master.out
13.检测是否安装成功
jps
http://192.168.5.30:8088/cluster
14.报错
hadoop-2.3.0安装错误
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platformhadoop本地库与系统版本不一致引起的错误
hadoop-2.2.0安装错误
在使用./sbin/start-dfs.sh或./sbin/start-all.sh启动时会报出这样如下警告：
Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library
/usr/local/hadoop-2.2.0/lib/native/libhadoop.so.1.0.0 which might have disabled
stack guard. The VM will try to fix the stack guard now.
....
Java: ssh: Could not resolve hostname Java: Name or service not known
HotSpot(TM): ssh: Could not resolve hostname HotSpot(TM): Name or service not
known
64-Bit: ssh: Could not resolve hostname 64-Bit: Name or service not known
....
这个问题的错误原因会发生在64位的操作系统上，原因是从官方下载的hadoop使用的本地库
文件(例如lib/native/libhadoop.so.1.0.0)都是基于32位编译的，运行在64位系统上就会出
现上述错误。
解决方法之一是在64位系统上重新编译hadoop，另一种方法是在hadoop-env.sh和yarn-env.sh中添加如下两行：
export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_HOME}/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"
15.hadoop使用端口说明
16.ss -an
页:
[1]