hadoop 出现问题以及解决方法

leonheart

　　

运行hadoop程序时出现以下问题。

09/12/13 19:47:12 INFO mapred.JobClient: Running job: job_200912131946_0001

09/12/13 19:47:13 INFO mapred.JobClient: map 0% reduce 0%

09/12/13 19:47:21 INFO mapred.JobClient: map 15% reduce 0%

09/12/13 19:47:24 INFO mapred.JobClient: map 23% reduce 0%

09/12/13 19:47:27 INFO mapred.JobClient: map 38% reduce 0%

09/12/13 19:47:30 INFO mapred.JobClient: map 46% reduce 0%

09/12/13 19:47:33 INFO mapred.JobClient: map 61% reduce 0%

09/12/13 19:47:36 INFO mapred.JobClient: map 69% reduce 0%

09/12/13 19:47:40 INFO mapred.JobClient: map 84% reduce 0%

09/12/13 19:47:46 INFO mapred.JobClient: map 100% reduce 0%09/12/13 19:58:18 INFO mapred.JobClient: Task Id : attempt_200912131946_0001_m_000000_0, Status : FAILED

Too many fetch-failures

09/12/13 20:01:27 WARN mapred.JobClient: Error reading task outputConnection timed out

09/12/13 20:04:36 WARN mapred.JobClient: Error reading task outputConnection timed out在/etc/hosts里面有一行是

201.139.56.34 TestMachine从网上查找，找到这个东西

1. Make sure that all the needed parameters are configured in hadoop-site.xml, and all the worker nodes should have the same content of this file.

2. URI for TaskTracker and HDFS should use hostname instead of IP address. I saw some instances of Hadoop cluster using IP address for the URI, they can start all the services and execute the jobs, but the task never finished successfully.

3. Check the file /etc/hosts on all the nodes and make sure that you’re binding the host name to its network IP, not the local one (127.0.0.1), don’t forget to check that all the nodes are able to communicate to the others using their hostname.

Anyway, it doesn’t make sense to me when Hadoop always try to resolve an IP address using the hostname. I consider this is a bug of Hadoop and hope they will solve it in next stable version.

然后我就把/etc/hosts里面的那一行改为

127.0.0.1 TestMachine

再运行程序就没有出现问题。

更多信息请查看 java进阶网 http://www.javady.com