【Hadoop Diary】调试篇<二>

sakko51150

　　自从上次调通hbase后很久没有碰hadoop了，今日想写一个mapreduce的小程序。于是先运行了下自带的wordcount示例程序，却报错了。
　　信息如下：



kevin@ubuntu:~/usr/hadoop/hadoop$ ./bin/hadoop jar hadoop-examples-1.2.1.jar wordcount readme.txt output
15/05/11 08:20:04 INFO input.FileInputFormat: Total input paths to process : 1
15/05/11 08:20:04 INFO util.NativeCodeLoader: Loaded the native-hadoop library
15/05/11 08:20:04 WARN snappy.LoadSnappy: Snappy native library not loaded
15/05/11 08:20:05 INFO mapred.JobClient: Running job: job_201505110806_0003
15/05/11 08:20:06 INFO mapred.JobClient:  map 0% reduce 0%
15/05/11 08:20:06 INFO mapred.JobClient: Task Id : attempt_201505110806_0003_m_000002_0, Status : FAILED
Error initializing attempt_201505110806_0003_m_000002_0:
java.io.IOException: Exception reading file:/~/usr/hadoop/hadoop/tmp/mapred/local/ttprivate/taskTracker/kevin/jobcache/job_201505110806_0003/jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:135)
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens(TokenCache.java:178)
at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1289)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1226)
at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2603)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.FileNotFoundException: File file:/~/usr/hadoop/hadoop/tmp/mapred/local/ttprivate/taskTracker/kevin/jobcache/job_201505110806_0003/jobToken does not exist.
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:402)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:255)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.(ChecksumFileSystem.java:125)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:436)
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:129)
... 5 more
　　错误是无法读取jobToken的文件，由于程序不会出错，推测是上次在配置hbase时修改hadoop的core-site.xml文件出错了。在stackoverflow上有一个同样的问题，有人建议删除 该配置文件中关于缓存文件目录的配置，因为这一部分是会被系统默认创建的，不需要手动配置。即下面这一部分：



1  
2  
3   hadoop.tmp.dir
4   /app/hadoop/tmp
5   A base for other temporary directories.
6  
　　删掉这个问题就解决了。关闭hadoop，重启hadoop。出现新问题 JobTracker is in safe mode：



15/05/11 08:47:40 ERROR security.UserGroupInformation: PriviledgedActionException as:kevin cause:org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.mapred.SafeModeException: JobTracker is in safe mode
at org.apache.hadoop.mapred.JobTracker.checkSafeMode(JobTracker.java:5188)
at org.apache.hadoop.mapred.JobTracker.getStagingAreaDir(JobTracker.java:3677)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1432)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1428)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426)
　　碰巧stackoverflow上也有相关的问题，应该是权限的问题，运行以下命令即可：



bin/hadoop dfsadmin -safemode leave
　　问题均解决，wordcount成功运行。