Ubuntu下Python访问Hive——thrift安装及测试

一、下载thrift：

wget http://mirror.bjtu.edu.cn/apache//thrift/0.8.0/thrift-0.9.0.tar.gz

二、安装依赖：

sudo apt-get install build-essential
sudo apt-get install bison flex
sudo apt-get install libboost-dev python-dev
sudo apt-get install autoconf automake libtool pkg-config
sudo apt-get install git

三、安装thrift：

tar -xzvf thrift-0.9.0.tar.gz
cd thrift-0.9.0
sudo ./configure --prefix=/host/java/thrift
sudo make && sudo make install

四、启动Hive Thrift Server  (>>hive --service  hiveserver)

要先启动hadoop 设置hdfs安全模式为leave

>>start-all.sh

>>hadoop dfsadmin -safemode leave

启动时报错……不要紧，运行python时候才会再有反应

Starting Hive Thrift Server
13/07/29 13:24:13 WARN conf.HiveConf: DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.

解决办法：

由于在0.10里面，hive.metastore.local参数已经取消，都使用hive.metastore.uris参数，若是hive.metastore.uris为空，就默认为local模式。

把hive.metastore.local参数删掉就行了。

五、测试代码编写以下：

testHive.py:

#!/usr/bin/env python

import sys

sys.path.append('/usr/local/hadoop/hive/lib/py')#'/usr/local/hadoop/hive是你的hive安装路径

from hive_service import ThriftHive

from hive_service.ttypes import HiveServerException

from thrift import Thrift

from thrift.transport import TSocket

from thrift.transport import TTransport

from thrift.protocol import TBinaryProtocol

 

def hiveExe(sql):

 

    try:

        transport = TSocket.TSocket('127.0.0.1', 10000) 

        transport = TTransport.TBufferedTransport(transport)

        protocol = TBinaryProtocol.TBinaryProtocol(transport)

        client = ThriftHive.Client(protocol)

        transport.open()

 

 

        client.execute(sql)

 

        print "The return value is : " 

        print client.fetchAll()

        print "............"

        transport.close()

    except Thrift.TException, tx:

        print '%s' % (tx.message)

 

if __name__ == '__main__':

    hiveExe("select * from testParttion")#t_afan_test是你的表名

六、执行

>>python testHive.py

七、结果

The return value is:

['[12,23,23,34]\t["what","are","this"]','[34,45,34,23,12]\t["who","am","i","are"]']