hive2.1.0 安装

 

参数配置地址

https://cwiki.apache.org/confluence/display/Hive/AdminManual+Configuration

 

一、配置HIVE_HOME 和PATH

HIVE_HOME=/opt/apache-hive-2.1.0-bin

PATH=$JAVA_BIN:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HIVE_HOME/bin:$PATH
export JAVA_HOME JAVA_BIN HADOOP_HOME HIVE_HOME  PATH CLASSPATH

二、拷贝hive-default.xml.template 为hive-site.xml

<property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>123456</value>
    <description>数据库链接密码</description>
  </property>
<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://192.168.8.145:3306/hive?createDatabaseIfNotExist=true</value>
    <description>
      JDBC connect string for a JDBC metastore.
      To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.
      For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.
    </description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    <description>驱动</description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>root</value>
    <description>用户名</description>
  </property>

  <property>
    <name>hive.exec.local.scratchdir</name>
    <value>/home/hive/local</value>
    <description>Local scratch space for Hive jobs</description>
  </property>
  <property>
    <name>hive.downloaded.resources.dir</name>
    <value>/home/hive/resources</value>
    <description>Temporary local directory for added resources in the remote file system.</description>
  </property>

 

三、 拷贝hive-env.sh.template 为hive-env.sh

修改

# Set HADOOP_HOME to point to a specific hadoop install directory
 HADOOP_HOME=/opt/hadoop-2.7.2

# Hive Configuration Directory can be controlled by:
 export HIVE_CONF_DIR=/opt/apache-hive-2.1.0-bin/conf

四、下载 mysql-connector-java-5.1.39.jar 到hive 的lib包中

 

 

5 初始化数据库

[root@master opt]# schematool -initSchema -dbType mysql
which: no hbase in (/opt/jdk1.7.0_79/bin:/opt/hadoop-2.7.2/bin:/opt/hadoop-2.7.2/sbin:/opt/apache-hive-2.1.0-bin/bin:/opt/jdk1.7.0_79/bin:/opt/hadoop-2.7.2/bin:/opt/hadoop-2.7.2/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/apache-hive-2.1.0-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop-2.7.2/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:      	 jdbc:mysql://192.168.8.145:3306/hive?createDatabaseIfNotExist=true
Metastore Connection Driver :  	 com.mysql.jdbc.Driver
Metastore connection User:     	 root
Tue Aug 23 18:09:10 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Starting metastore schema initialization to 2.1.0
Initialization script hive-schema-2.1.0.mysql.sql
Tue Aug 23 18:09:11 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Initialization script completed
Tue Aug 23 18:09:15 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
schemaTool completed

 

六、出现如下错误

Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
       	at org.apache.hadoop.fs.Path.initialize(Path.java:205)
       	at org.apache.hadoop.fs.Path.<init>(Path.java:171)
       	at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:631)
       	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:550)
       	at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:518)
       	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:705)
       	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)
       	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
       	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       	at java.lang.reflect.Method.invoke(Method.java:606)
       	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
       	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
       	at java.net.URI.checkPath(URI.java:1804)
       	at java.net.URI.<init>(URI.java:752)
       	at org.apache.hadoop.fs.Path.initialize(Path.java:202)
       	... 12 more

 

须要修改

<property>
    <name>hive.exec.local.scratchdir</name>
    <value>${system:java.io.tmpdir}/${system:user.name}</value>
    <description>Local scratch space for Hive jobs</description>
  </property>
 <property>
    <name>hive.downloaded.resources.dir</name>
    <value>${system:java.io.tmpdir}/${hive.session.id}_resources</value>
    <description>Temporary local directory for added resources in the remote file system.</description>
  </property>

改为

<property>
    <name>hive.exec.local.scratchdir</name>
    <value>/home/hive/local</value>
    <description>Local scratch space for Hive jobs</description>
  </property>
  <property>
    <name>hive.downloaded.resources.dir</name>
    <value>/home/hive/resources</value>
    <description>Temporary local directory for added resources in the remote file system.</description>
  </property>

输入hive

 

[root@master conf]# hive
which: no hbase in (/opt/jdk1.7.0_79/bin:/opt/hadoop-2.7.2/bin:/opt/hadoop-2.7.2/sbin:/opt/apache-hive-2.1.0-bin/bin:/opt/jdk1.7.0_79/bin:/opt/hadoop-2.7.2/bin:/opt/hadoop-2.7.2/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/apache-hive-2.1.0-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop-2.7.2/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

Logging initialized using configuration in jar:file:/opt/apache-hive-2.1.0-bin/lib/hive-common-2.1.0.jar!/hive-log4j2.properties Async: true
Tue Aug 23 18:37:52 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Aug 23 18:37:52 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Aug 23 18:37:52 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Aug 23 18:37:52 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Aug 23 18:37:55 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Aug 23 18:37:55 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Aug 23 18:37:55 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Aug 23 18:37:55 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. tez, spark) or using Hive 1.X releases.
hive> show databases;
OK
default
Time taken: 1.16 seconds, Fetched: 1 row(s)
hive>

 在dnfs中建立目录数据仓库目录

#hadoop fs -mkdir -r user/hive/warehouse

<property>

<name>hive.querylog.location</name>

<value>/usr/hive/log</value>

</property>

这个是用于存放hive相关日志的目录

其他的不用修改。

-------------------------------------------finish hive-site.xml

cp hive-log4j.properties.template  hive-log4j.proprties

vi hive-log4j.properties

hive.log.dir=

这个是当hive运行时，相应的日志文档存储到什么地方

（mine：hive.log.dir=/usr/hive/log/${user.name}）

hive.log.file=hive.log