centos7 安装 hadoop-cdh-2.5 on mesos

erwerwe

1.环境说明

master    192.168.0.223   mesos-master

slave     192.168.0.225   mesos-salve

2.环境准备
关闭防火墙
关闭selinux
两台机器修改主机名master/slave
设置hosts，能互相解析

3.master和slave配置ssh互信

这里配置hadoop用户双机互信，因为hadoop用hadoop用户启动

1 2 3 4 5 6 7 8 9

master   yum  -y install  sshpass   ssh-keygen 一路回车   ssh-copy-id -i ~/.ssh/id_rsa.pub hadoop@192.168.0.220 slave   yum  -y install  sshpass   ssh-keygen 一路回车   ssh-copy-id -i ~/.ssh/id_rsa.pub hadoop@192.168.0.201 测试ssh对方主机，不提示输入密码则OK

4.安装JDK

1 2	tar zxvf jdk-8u65-linux-x64.tar.gz mv jdk1.8.0_65  /usr/jdk

4.1设置环境变量

1 2 3 4 5

export JAVA_HOME=/usr/jdk export JRE_HOME=/usr/jdk/jre export CLASSPATH=.:$CLASSPATH:$JAVA_HOME/lib:$JRE_HOME/lib export PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin 执行 source  /etc/profile

4.2测试JDK

1	java -version #出现版本信息

5.安装mesos主从，看其他博客
安装完成，会在/usr/local/lib下生成一个libmesos.so文件

6.安装和配置Hadoop
master和slave

1 2 3 4 5 6 7 8 9 10 11 12 13 14

tar zxvf hadoop-2.5.0-cdh5.4.8.tar.gz mv hadoop-2.5.0-cdh5.4.8  /usr/hadoop cd  /usr/hadoop mkdir -p tmp cd /usr/hadoop/ mv bin bin-mapreduce2/ ln -s bin-mapreduce1  bin mv example  example-mapreduce2 ln -s example-mapreduce1  example cd etc/ mv hadoop hadoop-mapreduce2 ln -s hadoop-mapreduce1 hadoop

7.添加hadoop环境变量

1 2 3 4

vim /etc/profile export HADOOP_HOME=/usr/hadoop export PATH=$PATH:$HADOOP_HOME:$HADOOP_HOME/bin source /etc/profile

8.获取hadoop on mesos的jar包

1 2 3 4

yum -y install maven openjdk-7-jdk git git clone cd hadoop mvn package   #获取jar包,jar包会在target下

9.把获取的jar包放到hadoop安装目录下
master和slave

1	cp hadoop/target/hadoop-mesos-0.1.0.jar  /usr/hadoop/share/hadoop/common/lib/

10.配置hadoop on mesos
master和slave

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

vim  /usr/hadoop/etc/hadoop/mapred.site.xml <property>   <name>mapred.job.tracker</name>   <value>localhost:9001</value> </property> <property>   <name>mapred.jobtracker.taskScheduler</name>   <value>org.apache.hadoop.mapred.MesosScheduler</value> </property> <property>   <name>mapred.mesos.taskScheduler</name>   <value>org.apache.hadoop.mapred.JobQueueTaskScheduler</value> </property> <property>   <name>mapred.mesos.master</name>   <value>zk://192.168.0.223</value> </property> <property>   <name>mapred.mesos.executor.uri</name>   <value>hdfs://localhost:9000/hadoop-2.5.0-cdh5.2.0.tar.gz</value> </property>

11.给hadoop用户权限
master和slave

1	chown -R  hadoop:hadoop /usr/hadoop

12.在master上启动jobtracker，连接mesos

1 2	su hadoop MESOS_NATIVE_LIBRARY=/usr/local/lib/libmesos.so hadoop jobtracker

13.测试

输入 192.168.0.223:5050 看看框架里有没有hadoop