Mac搭建Hadoop源码阅读环境
1、本次Hadoop源码阅读环境使用的阅读工具是idea，Hadoop版本是2.7.3。需要安装的工具包括idea、jdk、maven、protobuf等2、jdk，使用的版本是1.8版，在jdk官网下载jdk-8u111-macosx-x64.dmg，点击安装，一路next。
3、idea安装，略
4、maven，使用的版本是3.3.9,下载apache-maven-3.3.9-bin.tar,解压：
tar -zxvfapache-maven-3.3.9-bin.tar
进入 Maven 安装根目录 cd apache-maven-3.3.9
进入 Maven配置文件目录 cd conf
编辑settings.xml文件 visettings.xml
设置Maven本地库的路径：这个路径要创建好。
<localRepository>/opt/PRG/apache-maven-3.3.9/repository</localRepository>
同时在加入阿里云的Maven库，参考：maven使用阿里云仓库
<mirror>
<id>nexus-aliyun</id>
<mirrorOf>*</mirrorOf>
<name>Nexus aliyun</name>
<url>http://maven.aliyun.com/nexus/content/groups/public</url>
</mirror>
设置Maven环境变量：
vi ~/.bash_profile
export MAVEN_HOME=/opt/PRG/apache-maven-3.3.9
export PATH=${PATH}:${MAVEN_HOME}/bin:
export MAVEN_OPTS="-Xmx2g -XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=512m"
export PATH
更新环境变量 source ~/.bash_profile
mvn -version 能看到相应的版本，则Maven安装成功。
5、安装protobuf，下载protobuf-2.5.0.tar.gz，下载地址： http://pan.baidu.com/s/1pJlZubT （网友提供）
解压：tar -zxvf protobuf-2.5.0.tar.gz
cd protobuf-2.5.0
依次执行，参考：http://blog.sina.com.cn/s/blog_7d1531ed0101fmc5.html
1 ./configure
2 make
3 make check
4 make install
如果报没有权限的错误，su root 切换到root执行。
最后输入protoc --version命令，如显示libprotoc 2.5.0则安装成功
6、编译Hadoop源码
在Hadoop官网下载hadoop-2.7.3-src.tar.gz源码
解压：tar -zxvfhadoop-2.7.3-src.tar.gz
cd hadoop-2.7.3-src
执行：
mvn package -Pdist -DskipTests -Dtar
最后是漫长的编译过程。。。
下载依赖jar和相关文件。。。
最后输出： Reactor Summary：
------------------------------------------------------------------------
Reactor Summary:
Apache Hadoop Main ................................. SUCCESS
Apache Hadoop Build Tools .......................... SUCCESS
Apache Hadoop Project POM .......................... SUCCESS
Apache Hadoop Annotations .......................... SUCCESS
Apache Hadoop Assemblies ........................... SUCCESS
Apache Hadoop Project Dist POM ..................... SUCCESS
Apache Hadoop Maven Plugins ........................ SUCCESS
Apache Hadoop MiniKDC .............................. SUCCESS
Apache Hadoop Auth ................................. SUCCESS
Apache Hadoop Auth Examples ........................ SUCCESS
Apache Hadoop Common ............................... SUCCESS
Apache Hadoop NFS .................................. SUCCESS
Apache Hadoop KMS .................................. SUCCESS
Apache Hadoop Common Project ....................... SUCCESS
Apache Hadoop HDFS ................................. SUCCESS
Apache Hadoop HttpFS ............................... SUCCESS [ 34.932 s]
Apache Hadoop HDFS BookKeeper Journal .............. SUCCESS [ 13.482 s]
Apache Hadoop HDFS-NFS ............................. SUCCESS
Apache Hadoop HDFS Project ......................... SUCCESS
hadoop-yarn ........................................ SUCCESS
hadoop-yarn-api .................................... SUCCESS [ 28.981 s]
hadoop-yarn-common ................................. SUCCESS [ 22.892 s]
hadoop-yarn-server ................................. SUCCESS
hadoop-yarn-server-common .......................... SUCCESS
hadoop-yarn-server-nodemanager ..................... SUCCESS
hadoop-yarn-server-web-proxy ....................... SUCCESS
hadoop-yarn-server-applicationhistoryservice ....... SUCCESS
hadoop-yarn-server-resourcemanager ................. SUCCESS [ 14.558 s]
hadoop-yarn-server-tests ........................... SUCCESS
hadoop-yarn-client ................................. SUCCESS
hadoop-yarn-server-sharedcachemanager .............. SUCCESS
hadoop-yarn-applications ........................... SUCCESS
hadoop-yarn-applications-distributedshell .......... SUCCESS
hadoop-yarn-applications-unmanaged-am-launcher ..... SUCCESS
hadoop-yarn-site ................................... SUCCESS
hadoop-yarn-registry ............................... SUCCESS
hadoop-yarn-project ................................ SUCCESS
hadoop-mapreduce-client ............................ SUCCESS
hadoop-mapreduce-client-core ....................... SUCCESS [ 14.922 s]
hadoop-mapreduce-client-common ..................... SUCCESS [ 11.181 s]
hadoop-mapreduce-client-shuffle .................... SUCCESS
hadoop-mapreduce-client-app ........................ SUCCESS
hadoop-mapreduce-client-hs ......................... SUCCESS
hadoop-mapreduce-client-jobclient .................. SUCCESS
hadoop-mapreduce-client-hs-plugins ................. SUCCESS
Apache Hadoop MapReduce Examples ................... SUCCESS
hadoop-mapreduce ................................... SUCCESS
Apache Hadoop MapReduce Streaming .................. SUCCESS
Apache Hadoop Distributed Copy ..................... SUCCESS
Apache Hadoop Archives ............................. SUCCESS
Apache Hadoop Rumen ................................ SUCCESS
Apache Hadoop Gridmix .............................. SUCCESS
Apache Hadoop Data Join ............................ SUCCESS
Apache Hadoop Ant Tasks ............................ SUCCESS
Apache Hadoop Extras ............................... SUCCESS
Apache Hadoop Pipes ................................ SUCCESS
Apache Hadoop OpenStack support .................... SUCCESS
Apache Hadoop Amazon Web Services support .......... SUCCESS [ 17.460 s]
Apache Hadoop Azure support ........................ SUCCESS
Apache Hadoop Client ............................... SUCCESS
Apache Hadoop Mini-Cluster ......................... SUCCESS
Apache Hadoop Scheduler Load Simulator ............. SUCCESS
Apache Hadoop Tools Dist ........................... SUCCESS
Apache Hadoop Tools ................................ SUCCESS
Apache Hadoop Distribution ......................... SUCCESS [ 29.114 s]
------------------------------------------------------------------------
BUILD SUCCESS
------------------------------------------------------------------------
Total time: 10:08 min
Finished at: 2017-01-02T11:41:11+08:00
Final Memory: 229M/959M
------------------------------------------------------------------------
没有报错，则说明编译成功。
7、把编译好的Hadoop源码导入Idea
源码路径
找到编译好的Hadoop，选择即可，next。
选择Maven工程
之后一路next即可，之后就可以开始阅读源码了。
页:
[1]