mac os x 编译spark-2.1.0 for hadoop-2.7.3

huijial

　　mac os x maven编译spark-2.1.0  for hadoop-2.7.3
　　1.官方文档中要求安装Maven 3.3.9+ 和Java 8 ;
　　2.执行   
　　export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=512m"
　　3.cd spark2.1.0源码根目录下
　　./build/mvn -Pyarn  -Phadoop-2.7  -Dhadoop.version=2.7.3  -Dscala-2.11      -Phive  -Phive-thriftserver    -DskipTests clean package
　　--

[INFO] ------------------------------------------------------------------------

[INFO] Reactor Summary:

[INFO]

[INFO] Spark Project Parent POM ........................... SUCCESS [02:10 min]

[INFO] Spark Project Tags ................................. SUCCESS [ 32.297 s]

[INFO] Spark Project Sketch ............................... SUCCESS [  3.362 s]

[INFO] Spark Project Networking ........................... SUCCESS [ 22.433 s]

[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [  6.130 s]

[INFO] Spark Project Unsafe ............................... SUCCESS [ 10.949 s]

[INFO] Spark Project Launcher ............................. SUCCESS [ 41.786 s]

[INFO] Spark Project Core ................................. SUCCESS [03:11 min]

[INFO] Spark Project ML Local Library ..................... SUCCESS [ 25.018 s]

[INFO] Spark Project GraphX ............................... SUCCESS [ 10.133 s]

[INFO] Spark Project Streaming ............................ SUCCESS [ 22.240 s]

[INFO] Spark Project Catalyst ............................. SUCCESS [01:05 min]

[INFO] Spark Project SQL .................................. SUCCESS [01:41 min]

[INFO] Spark Project ML Library ........................... SUCCESS [ 58.186 s]

[INFO] Spark Project Tools ................................ SUCCESS [  5.925 s]

[INFO] Spark Project Hive ................................. SUCCESS [01:11 min]

[INFO] Spark Project REPL ................................. SUCCESS [  3.290 s]

[INFO] Spark Project YARN Shuffle Service ................. SUCCESS [  4.925 s]

[INFO] Spark Project YARN ................................. SUCCESS [ 20.450 s]

[INFO] Spark Project Hive Thrift Server ................... SUCCESS [ 19.625 s]

[INFO] Spark Project Assembly ............................. SUCCESS [  2.035 s]

[INFO] Spark Project External Flume Sink .................. SUCCESS [ 15.382 s]

[INFO] Spark Project External Flume ....................... SUCCESS [  6.557 s]

[INFO] Spark Project External Flume Assembly .............. SUCCESS [  1.421 s]

[INFO] Spark Integration for Kafka 0.8 .................... SUCCESS [ 15.826 s]

[INFO] Spark Project Examples ............................. SUCCESS [ 14.344 s]

[INFO] Spark Project External Kafka Assembly .............. SUCCESS [  2.765 s]

[INFO] Spark Integration for Kafka 0.10 ................... SUCCESS [ 13.367 s]

[INFO] Spark Integration for Kafka 0.10 Assembly .......... SUCCESS [  2.907 s]

[INFO] Kafka 0.10 Source for Structured Streaming ......... SUCCESS [  6.780 s]

[INFO] Spark Project Java 8 Tests ......................... SUCCESS [  3.895 s]

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 15:32 min

[INFO] Finished at: 2017-05-01T14:11:28+08:00

[INFO] Final Memory: 88M/981M

[INFO] ------------------------------------------------------------------------

　　--

[WARNING] The requested profile "hadoop-2.8" could not be activated because it does not exist.

　　mvn 从github 下载 hadoop-2.8.0 的lib文件，这些lib文件还没人准备。。。因此下载不到，不会编译。
　　4切换到编译完的dev目录下执行下面命令打包
　　测试mvn编译成功后，可以用如下命令生成安装包：(当然直接执行生成安装包，但安全起见，还是先执测试一下)
　　./dev/make-distribution.sh --name custom-spark --tgz -Psparkr -Phadoop-2.7   -Phive -Phive-thriftserver  -Pyarn
　　参考：
　　http://spark.apache.org/docs/latest/building-spark.html