设为首页 收藏本站
查看: 973|回复: 0

[经验分享] CentOS6.4+hadoop-0.20.2安装实录

[复制链接]

尚未签到

发表于 2018-10-31 10:07:07 | 显示全部楼层 |阅读模式
  CentOS6.4+hadoop-0.20.2安装实录
  下载网址:
  http://mirror.bit.edu.cn/apache/
  资料网址
  http://hadoop.apache.org/docs/
  节点规划:
  10.10.1.131  hadoop1
  10.10.1.132  hadoop2
  10.10.1.133  hadoop3
  10.10.1.134  dog
  10.10.1.135  cat
  10.10.1.136  gangster
  一、解压安装
  在主节点上做
  验证JDK是否安装
  [root@hadoop1 ~]# java -version
  java version "1.7.0_09-icedtea"
  OpenJDK Runtime Environment (rhel-2.3.4.1.el6_3-x86_64)
  OpenJDK 64-Bit Server VM (build 23.2-b09, mixed mode)
  验证SSH是否安装
  [root@hadoop1 ~]# ssh -version
  OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010
  Bad escape character 'rsion'.
  各节点上都做
  [root@hadoop1 ~]#vi /etc/hosts
  10.10.1.131  hadoop1
  10.10.1.132  hadoop2
  10.10.1.133  hadoop3
  10.10.1.134  dog
  10.10.1.135  cat
  10.10.1.136  gangster
  [root@hadoop1 ~]# useradd hadoop
  [root@hadoop1 ~]# passwd hadoop
  [root@hadoop1 ~]# vi /etc/sysconfig/iptables
  -A INPUT -s 10.10.1.131 -j ACCEPT
  -A INPUT -s 10.10.1.132 -j ACCEPT
  -A INPUT -s 10.10.1.133 -j ACCEPT
  -A INPUT -s 10.10.1.170 -j ACCEPT
  -A INPUT -s 10.10.1.171 -j ACCEPT
  -A INPUT -s 10.10.1.172 -j ACCEPT
  二、安装hadoop
  把下载的hadoop-0.20.2.tar.gz文件上传到/home/hadoop目录下
  [hadoop@hadoop1 ~]$ tar xzvf hadoop-0.20.2.tar.gz
  -rw-r--r--.  1 hadoop hadoop 44575568 Feb 16 21:34 hadoop-0.20.2.tar.gz
  [hadoop@hadoop1 ~]$ cd hadoop-0.20.2
  [hadoop@hadoop1 hadoop-0.20.2]$ ll
  total 4872
  drwxr-xr-x.  2 hadoop hadoop    4096 Feb 16 21:37 bin
  -rw-rw-r--.  1 hadoop hadoop   74035 Feb 19  2010 build.xml
  drwxr-xr-x.  4 hadoop hadoop    4096 Feb 19  2010 c++
  -rw-rw-r--.  1 hadoop hadoop  348624 Feb 19  2010 CHANGES.txt
  drwxr-xr-x.  2 hadoop hadoop    4096 Feb 16 21:37 conf
  drwxr-xr-x. 13 hadoop hadoop    4096 Feb 19  2010 contrib
  drwxr-xr-x.  7 hadoop hadoop    4096 Feb 16 21:37 docs
  -rw-rw-r--.  1 hadoop hadoop    6839 Feb 19  2010 hadoop-0.20.2-ant.jar
  -rw-rw-r--.  1 hadoop hadoop 2689741 Feb 19  2010 hadoop-0.20.2-core.jar
  -rw-rw-r--.  1 hadoop hadoop  142466 Feb 19  2010 hadoop-0.20.2-examples.jar
  -rw-rw-r--.  1 hadoop hadoop 1563859 Feb 19  2010 hadoop-0.20.2-test.jar
  -rw-rw-r--.  1 hadoop hadoop   69940 Feb 19  2010 hadoop-0.20.2-tools.jar
  drwxr-xr-x.  2 hadoop hadoop    4096 Feb 16 21:37 ivy
  -rw-rw-r--.  1 hadoop hadoop    8852 Feb 19  2010 ivy.xml
  drwxr-xr-x.  5 hadoop hadoop    4096 Feb 16 21:37 lib
  drwxr-xr-x.  2 hadoop hadoop    4096 Feb 16 21:37 librecordio
  -rw-rw-r--.  1 hadoop hadoop   13366 Feb 19  2010 LICENSE.txt
  -rw-rw-r--.  1 hadoop hadoop     101 Feb 19  2010 NOTICE.txt
  -rw-rw-r--.  1 hadoop hadoop    1366 Feb 19  2010 README.txt
  drwxr-xr-x. 15 hadoop hadoop    4096 Feb 16 21:37 src
  drwxr-xr-x.  8 hadoop hadoop    4096 Feb 19  2010 webap
  三、配置主节点hadoop用户无密码访问从节点
  在MASTER节点上生成MASTER节点的ssh密钥对
  [root@hadoop1 ~]#su - hadoop
  [hadoop@hadoop1 ~]$ssh-keygen -t rsa
  Generating public/private rsa key pair.
  Enter file in which to save the key (/home/hadoop/.ssh/id_rsa):
  Created directory '/home/hadoop/.ssh'.
  Enter passphrase (empty for no passphrase):
  Enter same passphrase again:
  Your identification has been saved in /home/hadoop/.ssh/id_rsa.
  Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.
  The key fingerprint is:
  b9:5c:48:74:25:33:ac:9f:11:c9:77:5e:02:43:3b:ba hadoop@hadoop1.cfzq.com
  The key's randomart image is:
  +--[ RSA 2048]----+
  |        .o=+=.   |
  |       . .=+.oo .|
  |        .. ooo o |
  |       ..o.. ..  |
  |        S.oo     |
  |       . oo.     |
  |        o E      |
  |                 |
  |                 |
  +-----------------+
  把公钥拷贝到各SLAVE节点上
  [hadoop@hadoop1 .ssh]$ scp id_rsa.pub hadoop@hadoop2:~/master-key
  The authenticity of host 'hadoop2 (10.10.1.132)' can't be established.
  RSA key fingerprint is f9:47:3e:59:39:10:cd:7d:a4:5c:0d:ab:df:1f:14:21.
  Are you sure you want to continue connecting (yes/no)? yes
  Warning: Permanently added 'hadoop2,10.10.1.132' (RSA) to the list of known hosts.
  hadoop@hadoop2's password:
  id_rsa.pub                                                                                                                100%  405     0.4KB/s   00:00
  [hadoop@hadoop1 .ssh]$ scp id_rsa.pub hadoop@hadoop3:~/master-key
  The authenticity of host 'hadoop3 (10.10.1.133)' can't be established.
  RSA key fingerprint is f9:47:3e:59:39:10:cd:7d:a4:5c:0d:ab:df:1f:14:21.
  Are you sure you want to continue connecting (yes/no)? yes
  Warning: Permanently added 'hadoop3,10.10.1.133' (RSA) to the list of known hosts.
  hadoop@hadoop3's password:
  id_rsa.pub                                                                                                                100%  405     0.4KB/s   00:00
  [hadoop@hadoop1 .ssh]$ scp id_rsa.pub hadoop@cat:~/master-key
  The authenticity of host 'cat (10.10.1.171)' can't be established.
  RSA key fingerprint is f9:47:3e:59:39:10:cd:7d:a4:5c:0d:ab:df:1f:14:21.
  Are you sure you want to continue connecting (yes/no)? yes
  Warning: Permanently added 'cat,10.10.1.171' (RSA) to the list of known hosts.
  hadoop@cat's password:
  id_rsa.pub                                                                                                                100%  405     0.4KB/s   00:00
  [hadoop@hadoop1 .ssh]$ scp id_rsa.pub hadoop@dog:~/master-key
  The authenticity of host 'dog (10.10.1.170)' can't be established.
  RSA key fingerprint is f9:47:3e:59:39:10:cd:7d:a4:5c:0d:ab:df:1f:14:21.
  Are you sure you want to continue connecting (yes/no)? yes
  Warning: Permanently added 'dog,10.10.1.170' (RSA) to the list of known hosts.
  hadoop@dog's password:
  id_rsa.pub                                                                                                                100%  405     0.4KB/s   00:00
  [hadoop@hadoop1 .ssh]$ scp id_rsa.pub hadoop@gangster:~/master-key
  The authenticity of host 'gangster (10.10.1.172)' can't be established.
  RSA key fingerprint is f9:47:3e:59:39:10:cd:7d:a4:5c:0d:ab:df:1f:14:21.
  Are you sure you want to continue connecting (yes/no)? yes
  Warning: Permanently added 'gangster,10.10.1.172' (RSA) to the list of known hosts.
  hadoop@gangster's password:
  id_rsa.pub                                                                                                                100%  405     0.4KB/s   00:00
  在各SLAVE节点上:
  [hadoop@hadoop2 ~]$ mkdir .ssh
  [hadoop@hadoop2 ~]$ chmod 700 .ssh/
  [hadoop@hadoop2 ~]$ mv master-key .ssh/authorized_keys
  [hadoop@hadoop2 ~]$ cd .ssh/
  [hadoop@hadoop2 .ssh]$ chmod 600 authorized_keys
  [hadoop@hadoop3 ~]$ mkdir .ssh
  [hadoop@hadoop3 ~]$ chmod 700 .ssh/
  [hadoop@hadoop3 ~]$ mv master-key .ssh/authorized_keys
  [hadoop@hadoop3 ~]$ cd .ssh/
  [hadoop@hadoop3 .ssh]$ chmod 600 authorized_keys
  [hadoop@dog ~]$ mkdir .ssh
  [hadoop@dog ~]$ chmod 700 .ssh/
  [hadoop@dog ~]$ mv master-key .ssh/authorized_keys
  [hadoop@dog ~]$ cd .ssh/
  [hadoop@dog .ssh]$ chmod 600 authorized_keys
  [hadoop@cat ~]$ mkdir .ssh
  [hadoop@cat ~]$ chmod 700 .ssh/
  [hadoop@cat ~]$ mv master-key .ssh/authorized_keys
  [hadoop@cat ~]$ cd .ssh/
  [hadoop@cat .ssh]$ chmod 600 authorized_keys
  [hadoop@gangster ~]$ mkdir .ssh
  [hadoop@gangster ~]$ chmod 700 .ssh/
  [hadoop@gangster ~]$  mv master-key .ssh/authorized_keys
  [hadoop@gangster ~]$ cd .ssh/
  [hadoop@gangster .ssh]$ chmod 600 authorized_keys
  在MASTER节点上测试主节点无口令访问各从节点
  [hadoop@hadoop1 .ssh]$ ssh hadoop2
  Last login: Sat Feb 15 19:35:21 2014 from hadoop1
  [hadoop@hadoop2 ~]$ exit
  logout
  Connection to hadoop2 closed.
  [hadoop@hadoop1 .ssh]$ ssh hadoop3
  Last login: Sat Feb 15 21:57:38 2014 from hadoop1
  [hadoop@hadoop3 ~]$ exit
  logout
  Connection to hadoop3 closed.
  [hadoop@hadoop1 .ssh]$ ssh cat
  Last login: Sat Feb 15 14:33:50 2014 from hadoop1
  [hadoop@cat ~]$ exit
  logout
  Connection to cat closed.
  [hadoop@hadoop1 .ssh]$ ssh dog
  Last login: Sun Feb 16 20:41:19 2014 from hadoop1
  [hadoop@dog ~]$ exit
  logout
  Connection to dog closed.
  [hadoop@hadoop1 .ssh]$ ssh gangster
  Last login: Sat Feb 15 18:03:45 2014 from hadoop1
  [hadoop@gangster ~]$ exit
  logout
  Connection to gangster closed.
  二、配置运行参数
  几个主要参数要进行配置
  1.fs.default.name
  2.hadoop.tmp.dir
  3.mapred.job.tracker
  4.dfs.name.dir
  5.dfs.data.dir
  6.dfs.http.address
  [root@hadoop1 ~]# su - hadoop
  [hadoop@hadoop1 ~]$ cd hadoop-0.20.2/conf/
  [hadoop@hadoop1 conf]$ vi hadoop-env.sh
  export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk.x86_64
  export HADOOP_LOG_DIR=/var/log/hadoop
  在各节点:
  [hadoop@hadoop1 ~]# chmod 777 /var/log/hadoop
  [hadoop@hadoop2 ~]# chmod 777 /var/log/hadoop
  [hadoop@hadoop3 ~]# chmod 777 /var/log/hadoop
  测试JAVA_HOME设置是否正确
  [hadoop@hadoop1 bin]$ ./hadoop version
  Hadoop 0.20.2
  Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707
  Compiled by chrisdo on Fri Feb 19 08:07:34 UTC 2010
  [hadoop@hadoop1 conf]$ vi core-site.xml
  
  
  ##配置地址和端口
  fs.default.name
  hdfs://10.10.1.131:9000
  
  
  hdfs-site.xml中
  dfs.http.address提供web页面显示的地址和端口默认是50070,ip是namenode的ip
  dfs.data.dir是datanode机器上data数据存放的位置,没有则放到core-site.xml的tmp目录中
  dfs.name.dir是namenode机器上name数据存放的位置,没有则放到core-site.xml的tmp目录中
  [hadoop@hadoop1 hadoop-0.20.2]$ pwd
  /home/hadoop/hadoop-0.20.2
  [hadoop@hadoop1 hadoop-0.20.2]$ mkdir data
  [hadoop@hadoop1 hadoop-0.20.2]$ mkdir name
  [hadoop@hadoop1 hadoop-0.20.2]$ mkdir tmp
  [hadoop@hadoop1 conf]$ vi hdfs-site.xml
  
  
  dfs.name.dir
  /home/hadoop/hadoop-0.20.2/name
  
  
  dfs.data.dir
  /home/hadoop/hadoop-0.20.2/data
  
  
  dfs.http.address
  10.10.1.131:50071
  
  
  dfs.replication
  2
  The actual number of relications can be specified when the file is created.
  
  
  [hadoop@hadoop1 conf]$ vi mapred-site.xml
  
  
  mapred.job.tracker
  10.10.1.131:9001
  The host and port that the MapReduce job tracker runs at.
  
  [hadoop@hadoop1 conf]$ vi masters
  hadoop1
  [hadoop@hadoop1 conf]$ vi slaves
  hadoop2
  hadoop3
  四、向各datanodes节点复制hadoop
  [hadoop@hadoop1 ~]$ pwd
  /home/hadoop
  [hadoop@hadoop1 ~]$ scp -r hadoop-0.20.2 hadoop2:/home/hadoop/.
  [hadoop@hadoop1 ~]$ scp -r hadoop-0.20.2 hadoop3:/home/hadoop/.
  五、格式化分布式文件系统
  在namenode上
  [hadoop@hadoop1 bin]$ pwd
  /home/hadoop/hadoop-0.20.2/bin
  [hadoop@hadoop1 bin]$ ./hadoop namenode -format
  14/02/18 20:23:24 INFO namenode.NameNode: STARTUP_MSG:
  /************************************************************
  STARTUP_MSG: Starting NameNode
  STARTUP_MSG:   host = hadoop1.cfzq.com/10.10.1.131
  STARTUP_MSG:   args = [-format]
  STARTUP_MSG:   version = 0.20.2
  STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
  ************************************************************/
  14/02/18 20:23:38 INFO namenode.FSNamesystem: fsOwner=hadoop,hadoop
  14/02/18 20:23:38 INFO namenode.FSNamesystem: supergroup=supergroup
  14/02/18 20:23:38 INFO namenode.FSNamesystem: isPermissionEnabled=true
  14/02/18 20:23:39 INFO common.Storage: Image file of size 96 saved in 0 seconds.
  14/02/18 20:23:39 INFO common.Storage: Storage directory /home/hadoop/hadoop-0.20.2/data has been successfully formatted.
  14/02/18 20:23:39 INFO namenode.NameNode: SHUTDOWN_MSG:
  /************************************************************
  SHUTDOWN_MSG: Shutting down NameNode at hadoop1.cfzq.com/10.10.1.131
  ************************************************************/
  六、启动守护进程
  [hadoop@hadoop1 bin]$ pwd
  /home/hadoop/hadoop-0.20.2/bin
  [hadoop@hadoop1 bin]$ ./start-all.sh
  starting namenode, logging to /var/log/hadoop/hadoop-hadoop-namenode-hadoop1.cfzq.com.out
  hadoop3: starting datanode, logging to /var/log/hadoop/hadoop-hadoop-datanode-hadoop3.cfzq.com.out
  hadoop2: starting datanode, logging to /var/log/hadoop/hadoop-hadoop-datanode-hadoop2.cfzq.com.out
  hadoop1: starting secondarynamenode, logging to /var/log/hadoop/hadoop-hadoop-secondarynamenode-hadoop1.cfzq.com.out
  starting jobtracker, logging to /var/log/hadoop/hadoop-hadoop-jobtracker-hadoop1.cfzq.com.out
  hadoop3: starting tasktracker, logging to /var/log/hadoop/hadoop-hadoop-tasktracker-hadoop3.cfzq.com.out
  hadoop2: starting tasktracker, logging to /var/log/hadoop/hadoop-hadoop-tasktracker-hadoop2.cfzq.com.out
  在各节点上检查进程启动情况
  [hadoop@hadoop1 bin]$ /usr/lib/jvm/java-1.7.0-openjdk.x86_64/bin/jps
  30042 Jps
  29736 NameNode
  29885 SecondaryNameNode
  29959 JobTracker
  [hadoop@hadoop2 conf]$ jps
  13437 TaskTracker
  13327 DataNode
  13481 Jps
  [hadoop@hadoop3 ~]$ jps
  12117 Jps
  11962 DataNode
  12065 TaskTracker
  七、检测hadoop群集
  [hadoop@hadoop1 hadoop-0.20.2]$ pwd
  /home/hadoop/hadoop-0.20.2
  [hadoop@hadoop1 hadoop-0.20.2]$ mkdir input
  [hadoop@hadoop1 hadoop-0.20.2]$ cd input
  [hadoop@hadoop1 input]$ echo "hello world" >test1.txt
  [hadoop@hadoop1 input]$ echo "hello hadoop" >test2.txt
  [hadoop@hadoop1 input]$ ll
  total 8
  -rw-rw-r--. 1 hadoop hadoop 12 Feb 18 09:32 test1.txt
  -rw-rw-r--. 1 hadoop hadoop 13 Feb 18 09:33 test2.txt
  把input目录下的文件拷贝到hadoop系统里面去
  [hadoop@hadoop1 input]$ cd ../bin/
  [hadoop@hadoop1 bin]$ pwd
  /home/hadoop/hadoop-0.20.2/bin
  [hadoop@hadoop1 bin]$ ./hadoop dfs -put ../input in
  dfs参数:指示进行分布式文件系统的操作
  -put参数:把操作系统的文件输入到分布式文件系统
  ../input参数:操作系统的文件目录
  in参数:分布式文件系统的文件目录
  [hadoop@hadoop1 bin]$ ./hadoop dfs -ls ./in/*
  -rw-r--r--   2 hadoop supergroup         12 2014-02-18 10:11 /user/hadoop/in/test1.txt
  -rw-r--r--   2 hadoop supergroup         13 2014-02-18 10:11 /user/hadoop/in/test2.txt
  /user/hadoop/in表示用户hadoop的根目录下in目录,而非操作系统的目录。
  -ls参数:分布式文件系统中列出目录文件
  [hadoop@hadoop1 hadoop-0.20.2]$ pwd
  /home/hadoop/hadoop-0.20.2
  [hadoop@hadoop1 hadoop-0.20.2]$ bin/hadoop jar ../hadoop-0.20.2-examples.jar wordcount in out
  14/02/18 11:15:58 INFO input.FileInputFormat: Total input paths to process : 2
  14/02/18 11:15:59 INFO mapred.JobClient: Running job: job_201402180923_0001
  14/02/18 11:16:00 INFO mapred.JobClient:  map 0% reduce 0%
  14/02/18 11:16:07 INFO mapred.JobClient:  map 50% reduce 0%
  14/02/18 11:16:10 INFO mapred.JobClient:  map 100% reduce 0%
  14/02/18 11:16:19 INFO mapred.JobClient:  map 100% reduce 100%
  14/02/18 11:16:21 INFO mapred.JobClient: Job complete: job_201402180923_0001
  14/02/18 11:16:21 INFO mapred.JobClient: Counters: 17
  14/02/18 11:16:21 INFO mapred.JobClient:   Job Counters
  14/02/18 11:16:21 INFO mapred.JobClient:     Launched reduce tasks=1
  14/02/18 11:16:21 INFO mapred.JobClient:     Launched map tasks=2
  14/02/18 11:16:21 INFO mapred.JobClient:     Data-local map tasks=2
  14/02/18 11:16:21 INFO mapred.JobClient:   FileSystemCounters
  14/02/18 11:16:21 INFO mapred.JobClient:     FILE_BYTES_READ=55
  14/02/18 11:16:21 INFO mapred.JobClient:     HDFS_BYTES_READ=25
  14/02/18 11:16:21 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=180
  14/02/18 11:16:21 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=25
  14/02/18 11:16:21 INFO mapred.JobClient:   Map-Reduce Framework
  14/02/18 11:16:21 INFO mapred.JobClient:     Reduce input groups=3
  14/02/18 11:16:21 INFO mapred.JobClient:     Combine output records=4
  14/02/18 11:16:21 INFO mapred.JobClient:     Map input records=2
  14/02/18 11:16:21 INFO mapred.JobClient:     Reduce shuffle bytes=61
  14/02/18 11:16:21 INFO mapred.JobClient:     Reduce output records=3
  14/02/18 11:16:21 INFO mapred.JobClient:     Spilled Records=8
  14/02/18 11:16:21 INFO mapred.JobClient:     Map output bytes=41
  14/02/18 11:16:21 INFO mapred.JobClient:     Combine input records=4
  14/02/18 11:16:21 INFO mapred.JobClient:     Map output records=4
  14/02/18 11:16:21 INFO mapred.JobClient:     Reduce input records=4
  14/02/18 11:16:21 INFO mapred.JobClient:     Reduce input records=4
  jar参数:提交作业
  wordount参数:功能为wordcount
  in参数:原始数据位置
  out参数:输出数据位置
  [hadoop@hadoop1 hadoop-0.20.2]$ bin/hadoop dfs -ls
  Found 4 items
  drwxr-xr-x   - hadoop supergroup          0 2014-02-18 09:52 /user/hadoop/in
  drwxr-xr-x   - hadoop supergroup          0 2014-02-18 11:16 /user/hadoop/out
  [hadoop@hadoop1 hadoop-0.20.2]$ bin/hadoop dfs -ls ./out
  Found 2 items
  drwxr-xr-x   - hadoop supergroup          0 2014-02-18 11:15 /user/hadoop/out/_logs
  -rw-r--r--   2 hadoop supergroup         25 2014-02-18 11:16 /user/hadoop/out/part-r-00000
  [hadoop@hadoop1 hadoop-0.20.2]$ bin/hadoop dfs -cat ./out/part-r-00000
  hadoop  1
  hello   2
  world   1
  八、通过WEB了解hadoop的活动
  [hadoop@hadoop1 bin]$ netstat -all |grep :5
  getnameinfo failed
  tcp        0      0 *:57195                     *:*                         LISTEN
  tcp        0      0 hadoop1:ssh                 10.10.1.198:58000           ESTABLISHED
  tcp        0      0 *:50030                     *:*                         LISTEN
  tcp        0      0 *:50070                     *:*                         LISTEN
  tcp        0      0 *:59420                     *:*                         LISTEN
  tcp        0      0 *:50849                     *:*                         LISTEN
  tcp        0      0 *:50090                     *:*                         LISTEN
  ---监控jobtracker
  http://10.10.1.131:50030/jobtracker.jsp
  hadoop1 Hadoop Map/Reduce Administration
  Quick Links
  Scheduling Info
  Running Jobs
  Completed Jobs
  Failed Jobs
  Local Logs
  State: INITIALIZING
  Started: Mon Feb 17 21:33:23 CST 2014
  Version: 0.20.2, r911707
  Compiled: Fri Feb 19 08:07:34 UTC 2010 by chrisdo
  Identifier: 201402172133
  --------------------------------------------------------------------------------
  Cluster Summary (Heap Size is 117.94 MB/888.94 MB)
  Maps Reduces Total Submissions Nodes Map Task Capacity Reduce Task Capacity Avg. Tasks/Node Blacklisted Nodes
  0 0 0 0 0 0 - 0
  --------------------------------------------------------------------------------
  Scheduling Information
  Queue Name  Scheduling Information
  default N/A
  --------------------------------------------------------------------------------
  Filter (Jobid, Priority, User, Name)
  Example: 'user:smith 3200' will filter by 'smith' only in the user field and '3200' in all fields
  --------------------------------------------------------------------------------
  Running Jobs
  none
  --------------------------------------------------------------------------------
  Completed Jobs
  none
  --------------------------------------------------------------------------------
  Failed Jobs
  none
  --------------------------------------------------------------------------------
  Local Logs
  Log directory, Job Tracker History
  --------------------------------------------------------------------------------
  Hadoop, 2014.
  ---监控HDFS
  http://10.10.1.131:50070/dfshealth.jsp
  NameNode 'hadoop1:9000'
  Started:  Mon Feb 17 12:35:22 CST 2014
  Version:  0.20.2, r911707
  Compiled:  Fri Feb 19 08:07:34 UTC 2010 by chrisdo
  Upgrades:  There are no upgrades in progress.
  Browse the filesystem
  Namenode Logs
  --------------------------------------------------------------------------------
  Cluster Summary
  1 files and directories, 0 blocks = 1 total. Heap Size is 117.94 MB / 888.94 MB (13%)
  Configured Capacity : 0 KB
  DFS Used : 0 KB
  Non DFS Used : 0 KB
  DFS Remaining : 0 KB
  DFS Used% : 100 %
  DFS Remaining% : 0 %
  Live Nodes  : 0
  Dead Nodes  : 0
  There are no datanodes in the cluster
  --------------------------------------------------------------------------------
  NameNode Storage:
  Storage Directory Type State
  /tmp/hadoop-hadoop/dfs/name IMAGE_AND_EDITS Active
  --------------------------------------------------------------------------------
  Hadoop, 2014.


运维网声明 1、欢迎大家加入本站运维交流群:群②:261659950 群⑤:202807635 群⑦870801961 群⑧679858003
2、本站所有主题由该帖子作者发表,该帖子作者与运维网享有帖子相关版权
3、所有作品的著作权均归原作者享有,请您和我们一样尊重他人的著作权等合法权益。如果您对作品感到满意,请购买正版
4、禁止制作、复制、发布和传播具有反动、淫秽、色情、暴力、凶杀等内容的信息,一经发现立即删除。若您因此触犯法律,一切后果自负,我们对此不承担任何责任
5、所有资源均系网友上传或者通过网络收集,我们仅提供一个展示、介绍、观摩学习的平台,我们不对其内容的准确性、可靠性、正当性、安全性、合法性等负责,亦不承担任何法律责任
6、所有作品仅供您个人学习、研究或欣赏,不得用于商业或者其他用途,否则,一切后果均由您自己承担,我们对此不承担任何法律责任
7、如涉及侵犯版权等问题,请您及时通知我们,我们将立即采取措施予以解决
8、联系人Email:admin@iyunv.com 网址:www.yunweiku.com

所有资源均系网友上传或者通过网络收集,我们仅提供一个展示、介绍、观摩学习的平台,我们不对其承担任何法律责任,如涉及侵犯版权等问题,请您及时通知我们,我们将立即处理,联系人Email:kefu@iyunv.com,QQ:1061981298 本贴地址:https://www.yunweiku.com/thread-628831-1-1.html 上篇帖子: Hadoop虽然强大,但不是万能的 下篇帖子: ambari方式安装hadoop的hive组件遇到的问题
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

扫码加入运维网微信交流群X

扫码加入运维网微信交流群

扫描二维码加入运维网微信交流群,最新一手资源尽在官方微信交流群!快快加入我们吧...

扫描微信二维码查看详情

客服E-mail:kefu@iyunv.com 客服QQ:1061981298


QQ群⑦:运维网交流群⑦ QQ群⑧:运维网交流群⑧ k8s群:运维网kubernetes交流群


提醒:禁止发布任何违反国家法律、法规的言论与图片等内容;本站内容均来自个人观点与网络等信息,非本站认同之观点.


本站大部分资源是网友从网上搜集分享而来,其版权均归原作者及其网站所有,我们尊重他人的合法权益,如有内容侵犯您的合法权益,请及时与我们联系进行核实删除!



合作伙伴: 青云cloud

快速回复 返回顶部 返回列表