cloudera 部署Hadoop 更方便
注意：以下操作对应 rhel5/centos51 获取cdh3 yum 源
[*]wget -c http://archive.cloudera.com/redhat/cdh/cdh3-repository-1.0-1.noarch.rpm
2 安装下载的rpm 包
[*]yum --nogpgcheck localinstall cdh3-repository-1.0-1.noarch.rpm
//安装后将得到 cloudera-cdh3.repo 文件
[*]# ll /etc/yum.repos.d/
[*]total 40
[*]-rw-r--r-- 1 root root 1926 Aug 292011 CentOS-Base.repo
[*]-rw-r--r-- 1 root root631 Aug 292011 CentOS-Debuginfo.repo
[*]-rw-r--r-- 1 root root626 Aug 292011 CentOS-Media.repo
[*]-rw-r--r-- 1 root root 5390 Aug 292011 CentOS-Vault.repo
[*]-rw-r--r-- 1 root root201 Jul 142011 cloudera-cdh3.repo
3 导入 rpm key
[*]rpm --import http://archive.cloudera.com/redhat/cdh/RPM-GPG-KEY-cloudera
4 安装 hadoop 主程序
[*]yum install hadoop-0.20
5hadoop
[*]namenode
[*]datanode
[*]secondarynamenode
[*]jobtracker
[*]tasktracker
//比如安装 namenode 为: yum install hadoop-0.20-datanode ，不同角色安装不同服务
6 安装后 hadoop 目录
[*]//hadoop 配置文件目录
[*]# ll /etc/hadoop/
[*]total 8
[*]lrwxrwxrwx 1 root root 34 Feb 17 02:56 conf -> /etc/alternatives/hadoop-0.20-conf
[*]drwxr-xr-x 2 root root 4096 Feb 28 10:13 conf.empty
[*]drwxr-xr-x 2 root root 4096 Feb 28 10:15 conf.pseudo
[*]
[*]//hadoop 日志目录
[*]# ll /var/log/hadoop
[*]lrwxrwxrwx 1 root root 28 Feb 17 02:56 /var/log/hadoop -> /etc/alternatives/hadoop-log
[*]
[*]//hadoop 进程脚本
[*]# ll /etc/init.d/ | grep hadoop
[*]-rwxr-xr-x 1 root root3041 Feb 17 02:26 hadoop-0.20-datanode
[*]-rwxr-xr-x 1 root root3067 Feb 17 02:26 hadoop-0.20-jobtracker
[*]-rwxr-xr-x 1 root root3041 Feb 17 02:26 hadoop-0.20-namenode
[*]-rwxr-xr-x 1 root root3158 Feb 17 02:26 hadoop-0.20-secondarynamenode
[*]-rwxr-xr-x 1 root root3080 Feb 17 02:26 hadoop-0.20-tasktracker
7 修改配置文档 (hdfs 方面)
[*]//slaves 配置文件 namenode 上配置即可
[*]cat /etc/hadoop/conf/slaves
[*]
[*]datanode1
[*]datanode2
[*]
[*]//hdfs-site.xml 配置文件
[*]cat /etc/hadoop/conf/hdfs-site.xml
[*]
[*]
[*]
[*]
[*]
[*]
[*]
[*]
[*] dfs.replication
[*] 1
[*]
[*]
[*] dfs.permissions
[*] false
[*]
[*]
[*]
[*] dfs.safemode.extension
[*] 0
[*]
[*]
[*] dfs.safemode.min.datanodes
[*] 1
[*]
[*]
[*]
[*]
[*]
[*]
[*]dfs.data.dir
[*]/data/dfs/data
[*]
[*]
[*]
[*]hadoop.tmp.dir
[*]/data/dfs/tmp
[*]
[*]
[*]
[*]dfs.datanode.max.xcievers
[*]200000
[*]
[*]
[*]
[*]
[*]
[*]//core-site.xml 配置文件
[*]cat /etc/hadoop/conf/core-site.xml
[*]
[*]
[*]
[*]
[*]
[*]
[*]
[*]
[*]
[*]fs.default.name
[*]hdfs://namenode:8020
[*]
[*]
[*]
8 java 环境
[*]//下载安装jdk
[*]chmod +x jdk-6u26-linux-x64-rpm.bin
[*]./jdk-6u26-linux-x64-rpm.bin
[*]
[*]//编辑 profile
[*]vim /etc/profile
[*]export JAVA_HOME=/usr/java/jdk1.6.0_26
[*]export PATH=$JAVA_HOME/bin:$PATH
[*]
[*]//载入环境变量
[*]souce /etc/profile
9 启动hadoop 相应进程
[*]# /etc/init.d/hadoop-0.20-namenode start
[*]# jps
[*]5599 NameNode
[*]12889 Jps
参考
https://ccp.cloudera.com/display/CDHDOC/CDH3+Installation
页:
[1]