【hbase】使用thrift with python 访问HBase
HBase 版本： 0.98.6thrift 版本： 0.9.0
使用 thrift client with python 连接 HBase 报错：
1 Traceback (most recent call last):
2 File "D:\workspace\Python\py\helloworld.py", line 27, in <module>
3 tables = client.getTableNames()
4 File "E:\mazhongsoft\python\lib\hbase\Hbase.py", line 788, in getTableNames
5 return self.recv_getTableNames()
6 File "E:\mazhongsoft\python\lib\hbase\Hbase.py", line 798, in recv_getTableNames
7 (fname, mtype, rseqid) = self._iprot.readMessageBegin()
8 File "E:\mazhongsoft\python\lib\thrift\protocol\TBinaryProtocol.py", line 126, in readMessageBegin
9 sz = self.readI32()
10 File "E:\mazhongsoft\python\lib\thrift\protocol\TBinaryProtocol.py", line 203, in readI32
11 buff = self.trans.readAll(4)
12 File "E:\mazhongsoft\python\lib\thrift\transport\TTransport.py", line 58, in readAll
13 chunk = self.read(sz-have)
14 File "E:\mazhongsoft\python\lib\thrift\transport\TTransport.py", line 155, in read
15 self.__rbuf = StringIO(self.__trans.read(max(sz, self.DEFAULT_BUFFER)))
16 File "E:\mazhongsoft\python\lib\thrift\transport\TSocket.py", line 94, in read
17 raise TTransportException('TSocket read 0 bytes')
18 thrift.transport.TTransport.TTransportException: None
查找原因，过程如下：
1） 客户端代码
1 transport = TTransport.TBufferedTransport(TSocket('192.168.0.10', 9090))
2 protocol = TBinaryProtocol.TBinaryProtocol(transport)
3 client = Hbase.Client(protocol)
4 transport.open()
2） hbase-site.xml 配置如下
1 <property>
2 <name>hbase.regionserver.thrift.framed</name>
3 <value>true</value>
4 </property>
5 <property>
6 <name>hbase.regionserver.thrift.compact</name>
7 <value>true</value>
8 </property>
3） thrift server日志如下：（/var/log/hbase/hbase-hbase-thrift-<hostname>.log）
1 Wed Jan 14 14:54:43 CST 2015 Starting thrift on <hostname>
2 core file size (blocks, -c) 0
3 data seg size (kbytes, -d) unlimited
4 scheduling priority (-e) 0
5 file size (blocks, -f) unlimited
6 pending signals (-i) 191956
7 max locked memory (kbytes, -l) unlimited
8 max memory size (kbytes, -m) unlimited
9 open files (-n) 32768
10 pipe size (512 bytes, -p) 8
11 POSIX message queues (bytes, -q) 819200
12 real-time priority (-r) 0
13 stack size (kbytes, -s) 10240
14 cpu time (seconds, -t) unlimited
15 max user processes (-u) 1024
16 virtual memory (kbytes, -v) unlimited
17 file locks (-x) unlimited
18 INFO util.VersionInfo: HBase 0.98.6-cdh5.3.0
19 INFO util.VersionInfo: Subversion file:///data/jenkins/workspace/generic-package-rhel64-6-0/topdir/BUILD/hbase-0.98.6-cdh5.3.0 -r Unknown
20 INFO util.VersionInfo: Compiled by jenkins on Tue Dec 16 19:13:29 PST 2014
21 INFO thrift.ThriftServerRunner: Using default thrift server type
22 INFO thrift.ThriftServerRunner: Using thrift server type threadpool
23 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2-hbase.properties
24 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
25 INFO impl.MetricsSystemImpl: HBase metrics system started
26 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
27 INFO http.HttpServer: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
28 INFO http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context thrift
29 INFO http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
30 INFO http.HttpServer: Jetty bound to port 9095
31 INFO mortbay.log: jetty-6.1.26.cloudera.4
32 INFO mortbay.log: Started HttpServer$SelectChannelConnectorWithSafeStartup@0.0.0.0:9095
33 DEBUG thrift.ThriftServerRunner: Using compact protocol
34 DEBUG thrift.ThriftServerRunner: Using framed transport
35 INFO thrift.ThriftServerRunner: starting TBoundedThreadPoolServer on /0.0.0.0:9090; min worker threads=16, max worker threads=1000, max queued requests=1000
由以上红色文字部分可知，是因为thrift 的server端和client端的协议不匹配造成的。
解决方案有如下两个：
方案一：修改hbase-site.xml，禁用TFramedTransport和TCompactProtocol功能，即：
1 <property>
2 <name>hbase.regionserver.thrift.framed</name>
3 <value>false</value>
4 </property>
5 <property>
6 <name>hbase.regionserver.thrift.compact</name>
7 <value>false</value>
8 </property>
重启thrift 服务器： service hbase-thrift restart
方案二：修改客户端代码
1 transport = TFramedTransport(TSocket('192.168.0.10', 9090))
2 protocol = TCompactProtocol.TCompactProtocol(transport)
3 client = Hbase.Client(protocol)
4 transport.open()
如果报错未找到TFramedTransport和TCompactProtocol，请查看/usr/lib/python2.6/site-packages/thrift目录下有没有protocol/TCompactProtocol.py文件，检查transport/TTransport.py文件中有没有类TFramedTransport。如果没有，可以在thrift官网下载源码包 thrift-0.9.2.tar.gz，用其中的lib/py/src/覆盖/usr/lib/python2.6/site-packages/thrift/目录即可。
页:
[1]