Hadoop 离线安装 CDH5.1 第二章:cloudera manager与agent 安装

pqwsa

　　新建cloudera-scm用户 （做了分行，这个是一条linux命令，）
[hadoop@master cloudera-manager-5.1.0]$ sudo useradd --system　　
--home=/opt/cloudera-manager-5.1.0/run/cloudera-scm-server
　　
--no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
　　改/opt/cloudera-manager-5.1.0/etc/cloudera-scm-agent/config.ini 里的 server_port 和 server_host ,把server_port
改成ClouderaManager运行的主机名或ip, 默认server_port 端口是 7182
[hadoop@master cloudera-scm-agent]$ sudo vim config.ini　　
[General]
　　
# Hostname of the CM server.
　　
server_host=localhost
　　

　　
# Port that the CM server is listening on.
　　
server_port=7182
　　为Cloudera Manager 建立数据库，需要下载Mysql的JDBC驱动，本次从搜狐镜像上下载最新版
　　http://mirrors.sohu.com/mysql/Connector-J/mysql-connector-java-5.1.32.tar.gz
　　解压
之后找到mysql-connector-java-5.1.32-bin.jar放到/opt/cloudera-manager-5.1.0/share/cmf/lib/目录下
　　参考链接：
　　http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM5/latest/Cloudera-Manager-Installation-Guide/cm5ig_install_path_C.html
　　http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM5/latest/Cloudera-Manager-Installation-Guide/cm5ig_installing_configuring_dbs.html#cmig_topic_5_2_unique_1 数据库配置方法
　　[hadoop@master mysql-connector-java-5.1.32]$ sudo cp mysql-connector-java-5.1.32-bin.jar
/opt/cloudera-manager-5.1.0/share/cmf/lib/　　生成数据库:（关于密码这里有个小小的坑,如果你遇到过不去的地方，私信我）
[hadoop@master schema]$ pwd　　
/opt/cloudera-manager-5.1.0/share/cmf/schema
　　
[hadoop@master schema]$ sudo ./scm_prepare_database.sh mysql -h localhost -u root
　　
-phadoop --scm-host localhost scm scm scm
　　
Verifying that we can write to /opt/cloudera-manager-5.1.0/etc/cloudera-scm-server
　　
Creating SCM configuration file in /opt/cloudera-manager-5.1.0/etc/cloudera-scm-server
　　
Executing:  /usr/java/jdk1.7.0_67/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/opt/cloudera-manager-5.1.0/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /opt/cloudera-manager-5.1.0/etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
　　
[                          main] DbCommandExecutor              INFO  Successfully connected to database.
　　
All done, your SCM database is configured correctly!
　　启动SCM server
[hadoop@master init.d]$ sudo ./cloudera-scm-server start　　
Starting cloudera-scm-server:                              [  OK  ]
　　
[hadoop@master init.d]$ pwd
　　
/opt/cloudera-manager-5.1.0/etc/init.d
　　设置成开机启动
[hadoop@master init.d]$ sudo cp cloudera-scm-server /etc/init.d/cloudera-scm-server　　
[hadoop@master init.d]$ sudo chkconfig cloudera-scm-server on
　　将客户端脚本拷贝到客户机上
　　先SCP 编好的目录 到所有机器上 ，然后在每台机器上开启Agents端：
[hadoop@master opt]$ sudo scp -r /opt/cloudera-manager-5.1.0/  root@slave01.hadoop:/opt/　　
[hadoop@master opt]$ sudo scp -r /opt/cloudera-manager-5.1.0/  root@slave02.hadoop:/opt/
　　
[hadoop@master opt]$ sudo scp -r /opt/cloudera-manager-5.1.0/  root@slave03.hadoop:/opt/
　　
Starting cloudera-scm-agent:                               [FAILED]
　　排错中..... 看日志，发现少个什么libpython的东西
/opt/cloudera-manager-5.1.0/lib64/cmf/agent/build/env/bin/python: error　　
while loading shared libraries: libpython2.4.so.1.0: cannot open shared
　　
object file: No such file or directory
　　***,万能的google,发现有人也在抱怨，因为RHEL6默认指向是python2.6,RHEL5是python2.4况，https://groups.google.com/a/cloudera.org/forum/#!msg/scm-users/Nz8O8eRn1RA/Jk0PUOAhQUkJ
Hi Simon,　　
On Wed, Dec 21, 2011 at 6:44 AM, Simon  wrote:
　　
    Hello,
　　
    during the installation on all hosts process I get a few installation
　　
    errors:
　　
    1. the master (runs the manager) installs all hadoop packages, but
　　
    when it comes to Installing Clouder Manager agent package... it says
　　
    that it requires libpython2.4.so.1.0()(64bit), of course python2.4 is
　　
    not installed but 2.6 is, is there any workaround?
　　
    This also occurs if I try to install the agent manually via yum
　　
What OS are you running?  Is this an Amazon AMI?
　　
Usually, when folks report this error, they're using RHEL/CentOS 6 (which has python 2.6) but their yum repos are pointed to our RHEL/CentOS 5 repos (5 has python 2.4).
　　突然恍然大悟，我的问题应该也是版本的问题，翻到前面的步骤，果然，一个眼花我下载的是el5版本的cdh5.1
　　下载离线的cloudera manager安装包, 放置于/opt目录，并解压，改名
12[hadoop@master opt]$ sudo tar -xzvf cloudera-manager-el5-cm5.1.0_x86_64.tar.gz[hadoop@master opt]$ sudo mv cm-5.1.0 cloudera-manager-5.1.0　　发现了原因就好办了。删掉
[hadoop@master opt]$ sudo rm -rf /opt/cloudera　　
[hadoop@master opt]$ sudo rm -rf /opt/cloudera-manager-5.1.0/
　　
[hadoop@master opt]$ sudo rm -rf /opt/cloudera-manager-el5-cm5.1.0_x86_64.tar.gz
　　
[hadoop@master opt]$ ls
　　再来一遍，这次一定要小心看清el6 ，下到master.hadoop的/opt/里
wget　　
http://archive-primary.cloudera.com/cm5/cm/5/cloudera-manager-el6-cm5.1.1_x86_64.tar.gz
[hadoop@master opt]$ sudo tar -xzvf cloudera-manager-el6-cm5.1.0_x86_64.tar.gz　　装完查看
[hadoop@master opt]$ ls　　
cloudera  cloudera-manager-el6-cm5.1.1_x86_64.tar.gz  cm-5.1.1
　　删除我们之间建的用的cloudera-scm
#确认/etc/passwd里如下内容，删除后消失　　
cloudera-scm:x:497:497:Cloudera SCM User:/opt/cloudera-manager-5.1.0/run/cloudera-scm-server:/bin/false
　　

　　
[hadoop@master opt]$ sudo userdel cloudera-scm
　　
#重新创建cloudera-scm用户
　　
sudo useradd --system --home=/opt/cm-5.1.1/run/cloudera-scm-server --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
　　改改配置,解压的mysql-connector拷贝进去
[hadoop@master cm-5.1.1]$ sudo vim etc/cloudera-scm-agent/config.ini　　
[General]
　　
# Hostname of the CM server.
　　
server_host=master.hadoop
　　

　　
# Port that the CM server is listening on.
　　
server_port=7182
　　

　　

　　
[hadoop@master cm-5.1.1]$sudo cp /home/hadoop/mysql-connector-java-5.1.32/mysql-connector-java-5.1.32-bin.jar /opt/cm-5.1.1/share/cmf/lib/mysql-connector-java-5.1.32-bin.jar
　　前面SCM那一步，因为我们第一次创建过数据库，不用再删除数据库
[hadoop@master cm-5.1.1]$ sudo ./etc/init.d/cloudera-scm-server start　　
Starting cloudera-scm-server:                              [  OK  ]
　　删掉过去的开机启动项目,把新的加进去(没有对比过，估计el6和el5不一样吧)
[hadoop@master cm-5.1.1]$ sudo rm /etc/init.d/cloudera-scm-server[hadoop@master init.d]$ sudo cp cloudera-scm-server /etc/init.d/cloudera-scm-server　　
[hadoop@master init.d]$ sudo chkconfig cloudera-scm-server on
[hadoop@slave01 /]$ sudo /opt/cm-5.1.1/etc/init.d/cloudera-scm-agent start　　
Starting cloudera-scm-agent:                               [  OK  ]
　　
[hadoop@slave01 /]$ sudo chkconfig /etc/init.d/cloudera-scm-agent on
　　
error reading information on service /etc/init.d/cloudera-scm-agent: No such file or directory
　　
[hadoop@slave01 /]$ sudo chkconfig cloudera-scm-agent on
　　陆续把所有slave的服务都打开