Hadoop 2.x build native library on Mac os x
转自：http://www.micmiu.com/bigdata/hadoop/hadoop-build-native-library-on-mac-os-x/查阅有关官方介绍 http://wiki.apache.org/hadoop/HowToContribute 中有说明：Hadoop本地库只支持*nix平台，已经广泛使用在GNU/Linux平台上，但是不支持Cygwin和 Mac OS X 。搜索后发现已经有人给出了Mac OSX 系统下编译生成本地库的patch，下面详细介绍在Mac OSX 平台下编译Hadoop本地库的方法。
[一]、环境说明：
[*]Hadoop 2.2.0
[*]Mac OS X 10.9.1
详细的环境依赖（protoc、cmake 等）参见：Hadoop2.2.0源码编译 （http://www.micmiu.com/opensource/hadoop/hadoop-build-source-2-2-0/）中介绍。
[二]、Mac OSX 编译本地库的步骤：
1、checkout Hadoop 2.2.0的源码
1$svnco https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.2.0/2、patch 相关补丁
官方讨论地址：https://issues.apache.org/jira/browse/HADOOP-9648 里面有详细介绍
补丁下载链接：https://issues.apache.org/jira/secure/attachment/12617363/HADOOP-9648.v2.patch
1#切换到hadoop源码的根目录2$wget https://issues.apache.org/jira/secure/attachment/12617363/HADOOP-9648.v2.patch3$patch-p1 < HADOOP-9648.v2.patchps：如果要回退patch 执行：patch -RE -p1< HADOOP-9648.v2.patch 即可。
3、编译本地库
在Hadoop源码的根目录下执行编译本地库命令：
1$mvn package -Pdist,native -DskipTests -Dtar编译成功看到如下日志信息：
1------------------------------------------------------------------------2Reactor Summary:34Apache Hadoop Main ................................ SUCCESS 5Apache Hadoop Project POM ......................... SUCCESS 6Apache Hadoop Annotations ......................... SUCCESS 7Apache Hadoop Project Dist POM .................... SUCCESS 8Apache Hadoop Assemblies .......................... SUCCESS 9Apache Hadoop Maven Plugins ....................... SUCCESS 10Apache Hadoop MiniKDC ............................. SUCCESS 11Apache Hadoop Auth ................................ SUCCESS 12Apache Hadoop Auth Examples ....................... SUCCESS 13Apache Hadoop Common .............................. SUCCESS 14Apache Hadoop NFS ................................. SUCCESS 15Apache Hadoop Common Project ...................... SUCCESS 16Apache Hadoop HDFS ................................ SUCCESS 17Apache Hadoop HttpFS .............................. SUCCESS 18Apache Hadoop HDFS BookKeeper Journal ............. SUCCESS 19Apache Hadoop HDFS-NFS ............................ SUCCESS 20Apache Hadoop HDFS Project ........................ SUCCESS 21hadoop-yarn ....................................... SUCCESS 22hadoop-yarn-api ................................... SUCCESS 23hadoop-yarn-common ................................ SUCCESS 24hadoop-yarn-server ................................ SUCCESS 25hadoop-yarn-server-common ......................... SUCCESS 26hadoop-yarn-server-nodemanager .................... SUCCESS 27hadoop-yarn-server-web-proxy ...................... SUCCESS 28hadoop-yarn-server-resourcemanager ................ SUCCESS 29hadoop-yarn-server-tests .......................... SUCCESS 30hadoop-yarn-client ................................ SUCCESS 31hadoop-yarn-applications .......................... SUCCESS 32hadoop-yarn-applications-distributedshell ......... SUCCESS 33hadoop-mapreduce-client ........................... SUCCESS 34hadoop-mapreduce-client-core ...................... SUCCESS 35hadoop-yarn-applications-unmanaged-am-launcher .... SUCCESS 36hadoop-yarn-site .................................. SUCCESS 37hadoop-yarn-project ............................... SUCCESS 38hadoop-mapreduce-client-common .................... SUCCESS 39hadoop-mapreduce-client-shuffle ................... SUCCESS 40hadoop-mapreduce-client-app ....................... SUCCESS 41hadoop-mapreduce-client-hs ........................ SUCCESS 42hadoop-mapreduce-client-jobclient ................. SUCCESS 43hadoop-mapreduce-client-hs-plugins ................ SUCCESS 44Apache Hadoop MapReduce Examples .................. SUCCESS 45hadoop-mapreduce .................................. SUCCESS 46Apache Hadoop MapReduce Streaming ................. SUCCESS 47Apache Hadoop Distributed Copy .................... SUCCESS 48Apache Hadoop Archives ............................ SUCCESS 49Apache Hadoop Rumen ............................... SUCCESS 50Apache Hadoop Gridmix ............................. SUCCESS 51Apache Hadoop Data Join ........................... SUCCESS 52Apache Hadoop Extras .............................. SUCCESS 53Apache Hadoop Pipes ............................... SUCCESS 54Apache Hadoop OpenStack support ................... SUCCESS 55Apache Hadoop Client .............................. SUCCESS 56Apache Hadoop Mini-Cluster ........................ SUCCESS 57Apache Hadoop Scheduler Load Simulator ............ SUCCESS 58Apache Hadoop Tools Dist .......................... SUCCESS 59Apache Hadoop Tools ............................... SUCCESS 60Apache Hadoop Distribution ........................ SUCCESS 61------------------------------------------------------------------------62BUILD SUCCESS63------------------------------------------------------------------------64Total time:3:44.266s65Finished at: Fri Jan 17 10:06:17 CST 201466Final Memory: 66M/123M67------------------------------------------------------------------------68micmiu-mbp:trunkmicmiu$编译通过后可在 <HADOOP源码根目录>/hadoop-dist/target/hadoop-2.2.0/lib/ 目录下看到如下内容：
1micmiu-mbp:libmicmiu$ tree2.3|____.DS_Store4|____native5||____libhadoop.1.0.0.dylib6||____libhadoop.a7||____libhadoop.dylib8||____libhadooppipes.a9||____libhadooputils.a10||____libhdfs.0.0.0.dylib11||____libhdfs.a12||____libhdfs.dylib然后把 上面生成的本地库 copy到部署环境相应的位置，再建立软连接即可：
1$ls -slibhadoop.1.0.0.dylib libhadoop.so2$ls -slibhdfs.0.0.0.dylib libhdfs.so下载地址：http://yun.baidu.com/s/1c0jBZDQ
[三]、参考：
[*]http://wiki.apache.org/hadoop/HowToContribute
[*]http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-common/NativeLibraries.html
[*]https://issues.apache.org/jira/browse/HADOOP-9648
[*]https://issues.apache.org/jira/browse/HADOOP-3659
页:
[1]