Install hadoop-lzo-lib for Hadoop v1.1.1 on Mac OS X with Mountain Lion
Here is a quick guide to install hadoop lzo library, the install process is pretty easy actually.[*]
Install LZO with HomeBrew, LZO is a wonderful compression scheme to use with Hadoop. HomeBrew is prerequisite, visit https://github.com/mxcl/homebrew to see how to install HomeBrew.
brew install lzo
[*]Clone the hadoop-lzo source code to local:
git clone https://github.com/twitter/hadoop-lzo.git ~/code/hadoop-lzo && cd ~/code/hadoop-lzo
[*]Build hadoop-lzo to generate hadoop-lzo-lib as well as hadoop lzo native library, find the built artifacts in the target directory.
C_INCLUDE_PATH=/usr/local/Cellar/lzo/2.06/include \
LIBRARY_PATH=/usr/local/Cellar/lzo/2.06/lib \
mvn clean test package
Waiting for build complete. If something wrong, refer to https://github.com/twitter/hadoop-lzo for troubleshoting.
[*]Copy the built artifacts into hadoop lib directory or native library directory respectively.
[*]Once the libs are built and installed, you may want to add them to the class paths and library paths. That is, in hadoop-env.sh, set
export HADOOP_CLASSPATH=$HADOOP_HOME/lib/hadoop-lzo-xxxx.jar
export JAVA_LIBRARY_PATH=/path/to/hadoop-lzo-native-libs:/path/to/standard-hadoop-native-libs
[*]Note that there seems to be a bug in /path/to/hadoop/bin/hadoop; (refer to https://issues.apache.org/jira/browse/HADOOP-6453) . Comment out the line
JAVA_LIBRARY_PATH=''
See https://github.com/twitter/hadoop-lzo for further instruction if you have any question or leave a comment.
页:
[1]