解决Hadoop0.21.0 HDFS/MapReduce编译错误(ivy-resolve-common does not find hadoop-commo)

tangbinde

　　最近老大安排，要跟踪HDFS ConcatenateFile的前提条件；于是尝试用ant/ivy编译了一下hdfs，提示ivy找不到hadoop-common组件；于是又编译了一下mapreduce，提示ivy找不到hadoop-comm/hadoop-hdfs；编译hadoop-common，提示编译成功；在编译mapreduce错误如下：
　　ivy-resolve-common:
　　[ivy:resolve] 
　　[ivy:resolve] :: problems summary ::
　　[ivy:resolve] :::: WARNINGS
　　[ivy:resolve] module not found: org.apache.hadoop#hadoop-common;0.21.0
　　[ivy:resolve] ==== apache-snapshot: tried
　　[ivy:resolve]   https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common/0.21.0/hadoop-common-0.21.0.pom
　　[ivy:resolve]   -- artifact org.apache.hadoop#hadoop-common;0.21.0!hadoop-common.jar:
　　[ivy:resolve]   https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common/0.21.0/hadoop-common-0.21.0.jar
　　[ivy:resolve] ==== maven2: tried
　　[ivy:resolve]   http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.21.0/hadoop-common-0.21.0.pom
　　[ivy:resolve]   -- artifact org.apache.hadoop#hadoop-common;0.21.0!hadoop-common.jar:
　　[ivy:resolve]   http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.21.0/hadoop-common-0.21.0.jar
　　[ivy:resolve] module not found: org.apache.hadoop#hadoop-common-test;0.21.0
　　[ivy:resolve] ==== apache-snapshot: tried
　　[ivy:resolve]   https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common-test/0.21.0/hadoop-common-test-0.21.0.pom
　　[ivy:resolve]   -- artifact org.apache.hadoop#hadoop-common-test;0.21.0!hadoop-common-test.jar:
　　[ivy:resolve]   https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common-test/0.21.0/hadoop-common-test-0.21.0.jar
　　[ivy:resolve] ==== maven2: tried
　　[ivy:resolve]   http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common-test/0.21.0/hadoop-common-test-0.21.0.pom
　　[ivy:resolve]   -- artifact org.apache.hadoop#hadoop-common-test;0.21.0!hadoop-common-test.jar:
　　[ivy:resolve]   http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common-test/0.21.0/hadoop-common-test-0.21.0.jar
　　[ivy:resolve] module not found: org.apache.hadoop#hadoop-hdfs;0.21.0
　　[ivy:resolve] ==== apache-snapshot: tried
　　[ivy:resolve]   https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-hdfs/0.21.0/hadoop-hdfs-0.21.0.pom
　　[ivy:resolve]   -- artifact org.apache.hadoop#hadoop-hdfs;0.21.0!hadoop-hdfs.jar:
　　[ivy:resolve]   https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-hdfs/0.21.0/hadoop-hdfs-0.21.0.jar
　　[ivy:resolve] ==== maven2: tried
　　[ivy:resolve]   http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.21.0/hadoop-hdfs-0.21.0.pom
　　[ivy:resolve]   -- artifact org.apache.hadoop#hadoop-hdfs;0.21.0!hadoop-hdfs.jar:
　　[ivy:resolve]   http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.21.0/hadoop-hdfs-0.21.0.jar
　　[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
　　[ivy:resolve] ::          UNRESOLVED DEPENDENCIES         ::
　　[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
　　[ivy:resolve] :: org.apache.hadoop#hadoop-common;0.21.0: not found
　　[ivy:resolve] :: org.apache.hadoop#hadoop-common-test;0.21.0: not found
　　[ivy:resolve] :: org.apache.hadoop#hadoop-hdfs;0.21.0: not found
　　[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
　　[ivy:resolve] 
　　[ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
　　BUILD FAILED
　　/opt/hadoop-0.21.0/mapred/build.xml:1861: impossible to resolve dependencies:
　　resolve failed - see output for details
　　通过Google，发现原来这是Apache-Hadoop的BUG:
　　https://issues.apache.org/jira/browse/HDFS-1519(hdfs ivy配置bug)
　　主要原因是ivy配置问题：
　　hdfs的ivy在配置关联类库hadoop-common时候位置指定错误，具体修改如下：
　　hdfs/ivy.xml
　　<dependency org="org.apache.hadoop" name="hadoop-common" 
　　-----rev="${hadoop-common.version}" 
　　+++ rev="${hadoop-common.version}-SNAPSHOT"
　　conf="common->default"/>
　　hdfs/src/contrib/hdfsproxy/ivy.xml
　　<dependency org="org.apache.hadoop"
　　name="hadoop-common"
　　----       rev="${hadoop-common.version}"
　　+++      rev="${hadoop-common.version}-SNAPSHOT"
　　conf="common->default"/>
　　<dependency org="org.apache.hadoop"
　　name="hadoop-common-test"
　　---        rev="${hadoop-common.version}"
　　+++     rev="${hadoop-common.version}-SNAPSHOT"
　　conf="common->default"/>
　　hdfs/src/contrib/thriftfs/ivy.xml
　　<dependency org="org.apache.hadoop"
　　name="hadoop-common"
　　---        rev="${hadoop-common.version}"
　　+++     rev="${hadoop-common.version}-SNAPSHOT"
　　conf="common->default"/>
　　mapreduce的ivy在配置关联类库hadoop-common/hadoop-dfs时候位置指定错误，修改方法与hdfs类似：
　　1.修改mapreduce project里的ivy.xml(多个)
　　在eclipse里search file(ivy.xml):
　　(1)rev="${hadoop-common.version}"  replace rev="${hadoop-common.version}-SNAPSHOT"
　　(2)rev="${hadoop-hdfs.version}"  replace rev="${hadoop-hdfs.version}-SNAPSHOT"
　　2.修改buid.xml
　　<unzip
　　-- --      src="${common.ivy.lib.dir}/hadoop-hdfs-${hadoop-hdfs.version}.jar"
　　+++     src="${common.ivy.lib.dir}/hadoop-hdfs-${hadoop-hdfs.version}-SNAPSHOT.jar"
　　dest="${build.dir}">