hadoop2.7.2修改配置文件，配置linux java环境变量

jhhll1

1.修改core-site.xml，加上
<property>
        <name>fs.defaultFS</name>
        <value>hdfs://backup02:9000</value>
    </property>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>file:/home/zhongml/hadoop-2.7.2/tmp</value>
    </property>
    <property>
        <name>io.file.buffer.size</name>
        <value>131702</value>
    </property>
2.修改hdfs-site.xml，加上
<property>
        <name>dfs.namenode.name.dir</name>
        <value>file:/home/zhongml/hadoop-2.7.2/hdfs/name</value>
    </property>
    <property>
        <name>dfs.datanode.data.dir</name>
        <value>file:/home/zhongml/hadoop-2.7.2/hdfs/data</value>
    </property>
    <property>
        <name>dfs.replication</name>
        <value>2</value>
    </property>
    <property>
        <name>dfs.namenode.secondary.http-address</name>
        <value>backup02:9001</value>
    </property>
    <property>
        <name>dfs.webhdfs.enabled</name>
        <value>true</value>
    </property>
3.先复制一个mapred-site.xml
    cp mapred-site.xml.template mapred-site.xml
修改mapred-site.xml，加上
<property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
    <property>
        <name>mapreduce.jobhistory.address</name>
        <value>backup02:10020</value>
    </property>
    <property>
        <name>mapreduce.jobhistory.webapp.address</name>
        <value>backup02:19888</value>
    </property>
4.修改yarn-site.xml，加上
<property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    <property>
        <name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name>
        <value>org.apache.hadoop.mapred.ShuffleHandler</value>
    </property>
    <property>
        <name>yarn.resourcemanager.address</name>
        <value>backup02:8032</value>
    </property>
    <property>
        <name>yarn.resourcemanager.scheduler.address</name>
        <value>backup02:8030</value>
    </property>
    <property>
        <name>yarn.resourcemanager.resource-tracker.address</name>
        <value>backup02:8031</value>
    </property>
    <property>
        <name>yarn.resourcemanager.admin.address</name>
        <value>backup02:8033</value>
    </property>
    <property>
        <name>yarn.resourcemanager.webapp.address</name>
        <value>backup02:8088</value>
    </property>
    <property>
        <name>yarn.nodemanager.resource.memory-mb</name>
        <value>768</value>
    </property>
5.配置/home/zhongml/hadoop-2.7.2/etc/hadoop目录下hadoop-env.sh、yarn-env.sh的JAVA_HOME
6.配置/home/zhongml/hadoop-2.7.2/etc/hadoop目录下slaves
7.验证启动是否成功
bin/hadoop fs -ls
http://localhost:50030(MapReduce的页面)
http://localhost:50070(HDFS的页面)
8.如果是64位的linux，需要覆盖native
9.配置java环境变量/etc/profile
先卸载linux自带的jdk，
执行以下命令查看需要卸载的选项:
rpm -qa | grep gcj
执行卸载操作:
yum -y remove java-1.4.2-gcj-compat-1.4.2.0-40jpp.115
将以下内容添加到/etc/profile末尾
JAVA_HOME=/jdk/jdk1.8.0_101/
PATH=$JAVA_HOME/bin:$PATH
CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export JAVA_HOME
export PATH
export CLASSPATH
注意:装好hadoop环境必须关闭防火墙:
关闭防火墙服务
[iyunv@cluster3 hadoop]# service iptables stop
关闭开机自动启动
[iyunv@cluster3 hadoop]# chkconfig iptables off




安装以下软件:
yum install svn
yum install autoconfautomake libtool cmake
yum install ncurses-devel
yum install openssl-devel
yum install gcc*