1.Hive簡(jiǎn)介

? 起源自facebook由Jeff Hammerbacher領(lǐng)導(dǎo)的團(tuán)隊(duì)
? 構(gòu)建在Hadoop上的數(shù)據(jù)倉(cāng)庫(kù)框架
? 設(shè)計(jì)目的是讓SQL技能良好，但Java技能較弱的分析師可以查詢海量數(shù)據(jù)
? 2008年facebook把hive項(xiàng)目貢獻(xiàn)給Apache

?Hive的組件與體系架構(gòu)

? 用戶接口：shell, thrift, web等
? Thrift服務(wù)器
? 元數(shù)據(jù)庫(kù)“Derby, Mysql等
? 解析器
? Hadoop

?Hive安裝模式

? 內(nèi)嵌模式：元數(shù)據(jù)保持在內(nèi)嵌的Derby模式，只允許一個(gè)會(huì)話連接(默認(rèn))
? 本地獨(dú)立模式：在本地安裝Mysql，把元數(shù)據(jù)放到Mysql內(nèi)
? 遠(yuǎn)程模式：元數(shù)據(jù)放置在遠(yuǎn)程的Mysql數(shù)據(jù)庫(kù)

2.配置Mysql

1:copy?mysql-connector-java-5.1.6-bin.jar到$HIVE_HOME/lib

[jifeng@jifeng02 hadoop]$ ls 7287OS_Code hadoop-1.2.1.tar.gz hive-0.12.0-bin tmp hadoop-1.2.1 hadoop-2.4.1-src.tar.gz hive-0.12.0-bin.tar.gz hadoop-1.2.1-bin.tar.gz hadoop-2.4.1.tar.gz mysql-connector-java-5.1.6-bin.jar [jifeng@jifeng02 hadoop]$ cp mysql-connector-java-5.1.6-bin.jar hive-0.12.0-bin/lib
2:修改$HIVE_HOME/conf/hive-site.xml

<property><name>javax.jdo.option.ConnectionURL</name><value>jdbc:mysql://jifengsql:3306/hive?createDatabaseIfNotExist=true</value><description>JDBC connect string for a JDBC metastore</description> </property><property><name>javax.jdo.option.ConnectionDriverName</name><value>com.mysql.jdbc.Driver</value><description>Driver class name for a JDBC metastore</description> </property><property><name>javax.jdo.PersistenceManagerFactoryClass</name><value>org.datanucleus.api.jdo.JDOPersistenceManagerFactory</value><description>class implementing the jdo persistence</description> </property><property><name>javax.jdo.option.DetachAllOnCommit</name><value>true</value><description>detaches all objects from session so that they can be used after transaction is committed</ description> </property><property><name>javax.jdo.option.NonTransactionalRead</name><value>true</value><description>reads outside of transactions</description> </property><property><name>javax.jdo.option.ConnectionUserName</name><value>dss</value><description>username to use against metastore database</description> </property><property><name>javax.jdo.option.ConnectionPassword</name><value>jifeng</value><description>password to use against metastore database</description> </property>
3:啟動(dòng)hive

[jifeng@jifeng02 hive-0.12.0-bin]$ hiveLogging initialized using configuration in jar:file:/home/jifeng/hadoop/hive-0.12.0-bin/lib/hive-common-0.12.0.jar!/hive-log4j.properties hive> show tables; FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient hive> quit;報(bào)錯(cuò)，網(wǎng)上查詢后說(shuō)是沒(méi)把?mysql的jar包mysql-connector-java-5.1.10-bin.jar，放在hive安裝目錄的lib下

把?mysql-connector-java-5.1.6-bin.jar 替換成mysql-connector-java-5.1.10-bin.jar還是不行。

檢查 mysql發(fā)現(xiàn)連接不上，換個(gè)虛擬機(jī)上mysql

[dss@localhost ~]$ mysql -u root -p Enter password: --root 登陸mysql Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 70 Server version: 5.6.16 MySQL Community Server (GPL)Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql> create database hive; Query OK, 1 row affected (0.01 sec) --創(chuàng)建hive庫(kù)mysql> GRANT all ON hive.* TO dss@'%' IDENTIFIED BY 'abc123'; Query OK, 0 rows affected (0.03 sec)--給dss用戶對(duì)hive庫(kù)授權(quán)mysql> flush privileges; Query OK, 0 rows affected (0.02 sec)--刷新系統(tǒng)權(quán)限表mysql> set globalbinlog_format='MIXED'; ERROR 1193 (HY000): Unknown system variable 'globalbinlog_format' mysql> alter database hive character set latin1 ; Query OK, 1 row affected (0.00 sec)--更新字符集

再次啟動(dòng)

[jifeng@jifeng02 hive-0.12.0-bin]$ hiveLogging initialized using configuration in jar:file:/home/jifeng/hadoop/hive-0.12.0-bin/lib/hive-common-0.12.0.jar!/hive-log4j.properties hive> show tables; OK Time taken: 6.273 seconds hive>
沒(méi)有錯(cuò)誤了。

4.Hive的運(yùn)行模式即任務(wù)的執(zhí)行環(huán)境

1啟動(dòng)hive?命令行模式：

1：直接輸入#hive的執(zhí)行程序，

2：或者輸入?#hive --service cli

?

l??分為本地與集群兩種

我們可以通過(guò)mapred.job.tracker?來(lái)指明

設(shè)置方式：

hive >?

SET mapred.job.tracker=local

?

2.hive驗(yàn)證啟動(dòng)的方法

l??1、hive web界面的(端口號(hào)9999)?啟動(dòng)方式

#hive --service hwi?

[jifeng@jifeng02 hive-0.12.0-bin]$ hive --service cliLogging initialized using configuration in jar:file:/home/jifeng/hadoop/hive-0.12.0-bin/lib/hive-common-0.12.0.jar!/hive-log4j.properties hive> quit; [jifeng@jifeng02 hive-0.12.0-bin]$ hive --service hwi 15/08/17 15:17:10 INFO hwi.HWIServer: HWI is starting up 15/08/17 15:17:10 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 15/08/17 15:17:10 INFO mortbay.log: jetty-6.1.26 15/08/17 15:17:10 INFO mortbay.log: Extract /home/jifeng/hadoop/hive-0.12.0-bin/lib/hive-hwi-0.12.0.war to /tmp/Jetty_0_0_0_0_9999_hive.hwi.0.12.0.war__hwi__ow27i/webapp 15/08/17 15:17:11 INFO mortbay.log: Started SocketConnector@0.0.0.0:9999

用于通過(guò)瀏覽器來(lái)訪問(wèn)hive

http://jifeng02:9999/hwi/

l??2、hive?遠(yuǎn)程服務(wù)(端口號(hào)10000)?啟動(dòng)方式

#hive --service hiveserver?

總結(jié)

以上是生活随笔為你收集整理的Hive学习和配置Mysql的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺(jué)得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。