HBase配置LZO压缩

云深处

　　系统：  gentoo
　　HDFS:  hadoop:hadoop-0.20.2-cdh3u0

HBase: HBase:0.90.3
　　安装过程：
　　1、安装lzo
　　emerge dev-libs/lzo


2.1 下载
　　download from:

wget http://www.apache.org/dist//hbase/hbase-0.90.3/hbase-0.90.3.tar.gz
wget http://hadoop-gpl-compression.apache-extras.org.codespot.com/files/hadoop-gpl-compression-0.1.0-rc0.tar.gz
　　

2.2 编译
　　tar -zxvf hadoop-gpl-compression-0.1.0-rc0.tar.gz

cd hadoop-gpl-compression-0.1.0

cp $HADOOP_HOME//hadoop-core-0.20.2-cdh3u0.jar lib/
　　nano -w src/java/com/hadoop/compression/lzo/LzoCompressor.java

......
import org.apache.hadoop.conf.Configuration;
.....
public void reinit(Configuration conf) {
// do nothing
}  
　　

ant compile-native tar
　　2.3 拷贝需要的文件

cp build/hadoop-gpl-compression-0.1.0-dev.jar $HADOOP_HOME/lib/

cp build/hadoop-gpl-compression-0.1.0-dev.jar $HBASE_HOME/lib/


rsync -auPv lib/native $HADOOP_HOME/lib/

rsync -auPv lib/native $HBASE_HOME/lib/
　　3.修改配置重启HBase和Hadoop

nano core-site.xml

<property>
<name>io.compression.codecs</name>
<value>com.hadoop.compression.lzo.LzoCodec,com.hadoop.compression.lzo.LzopCodec</value>
</property>
<property>
<name>io.compression.codec.lzo.class</name>
<value>com.hadoop.compression.lzo.LzoCodec</value>
</property>   
 　　4 测试是否成功

bin/hbase   org.apache.hadoop.hbase.util.CompressionTest hdfs://test1.namenode.uc.uuwatch.com:9000/test_path lzo



5 参考资料
　　

http://wiki.apache.org/hadoop/UsingLzoCompression

http://wiki.apache.org/hadoop/Hbase/HdfsSyncSupport
　　 http://koven2049.iyunv.com/blog/983502