hive

Hive安装部署

Hive安装

安装启动

• 解压文件

tar -zxvf   hive-0.13.1-cdh5.3.6.tar.gz -C /opt/cdh3.5.6/

• 配置文件hive-env.sh

# Set HADOOP_HOME to point to a specific hadoop install directory
 HADOOP_HOME=/opt/cdh3.5.6/hadoop-2.5.0-cdh5.3.6

# Hive Configuration Directory can be controlled by:
 export HIVE_CONF_DIR=/opt/cdh3.5.6/hive-0.13.1-cdh5.3.6/conf

• 创建工作目录

  $ /opt/cdh3.5.6/hadoop-2.5.0-cdh5.3.6/bin/hdfs dfs -mkdir /tmp
  $ /opt/cdh3.5.6/hadoop-2.5.0-cdh5.3.6/bin/hdfs dfs -mkdir -p /user/hive/warehouse
  $ /opt/cdh3.5.6/hadoop-2.5.0-cdh5.3.6/bin/hdfs dfs -chmod g+w /tmp
  $ /opt/cdh3.5.6/hadoop-2.5.0-cdh5.3.6/bin/hdfs dfs -chmod g+w  /user/hive/warehouse

• 启动hive

 $HIVE_HOME/bin/hive

数据测试

• 创建表格

create table student(id int,name string)ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'; 

• 加载数据

load data local inpath '/opt/datas/student.txt'into table student;

• 查询数据

select * from student ;

• 查询表格部分字段的内容

select name from student ;

• 加载数据并覆盖之前的数据

load data local inpath '/opt/datas/student.txt' overwrite into table student;

安装MySQL

准备工作

• 修改源

mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup

NOTE:163源下载地址：http://mirrors.163.com/.help/CentOS6-Base-163.repo

• 运行以下命令生成缓存

yum clean all
yum makecache

在线安装MySQL

• 在线安装

sudo yum install mysql-server -y

• 查看mysql服务是不是开启

sudo service mysqld status

• 切换到root用户,设置mysql密码

/usr/bin/mysqladmin -u root password '123456'

• 连接mysql

mysql -uroot -p123456

• 设定开机启动

sudo chkconfig mysqld on

• 查看状态

sudo chkconfig --list|grep mysqld

Hive连接MySQL的配置

• hive-site.xml

<configuration>
    <!-- metastore-->
    <property>
        <name>javax.jdo.option.ConnectionURL</name>
        <value>jdbc:mysql://hadoop-senior01.grc.com/metastore?createDatabaseIfNotExist=true</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionDriverName</name>
        <value>com.mysql.jdbc.Driver</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>root</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionPassword</name>
        <value>123456</value>
    </property>
</configuration>

• mysql jar包设置

• 解压mysql驱动

tar -zxvf mysql-connector-java-5.1.27.tar.gz 

*　复制到hive/lib下

cp -r mysql-connector-java-5.1.27-bin.jar /opt/cdh3.5.6/hive-0.13.1-cdh5.3.6/lib/

设置用户连接权限

• 查询用户信息

show databases ;
use mysql ;
select user,host,password from user;

update user set host='%' where user='root' and host='localhost';

• 删除用户信息

delete from user where user='root' and host='127.0.0.1';
delete from user where user='root' and host='hadoop-senior02.grc.com ';
delete from user where user='root' and host='localhost';
delete from user where host='localhost';
delete from user where host='hadoop-senior02.grc.com';

• 查询用户信息

select user,host,password from user;

• 刷新信息

flush privileges;

• 启动hive后查看mysql数据库

Hive常用属性配置

Hive数据仓库的位置

• hive-site.xml

<property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/user/hive/warehouse</value>
</property>

注意权限

bin/hdfs dfs -mkdir -p /user/hive/warehouse

bin/hdfs dfs -chmod g+x /user/hive/warehouse

Hive运行日志信息位置

• 在hive根目录下创建文件夹

mkdir logs

• 在log4j.properties中配置

添加：

hive.log.dir=/opt/cdh3.5.6/hive-0.13.1-cdh5.3.6/logs
hive.log.file=hive.log

指定hive运行时显示的log日志的级别

在log4j.properties配置

hive.root.logger=DEBUG,DRFA

在cli命令行上显示当前数据库，以及查询表的行头信息

• hive-site.xml

<property>
    <name>hive.cli.print.header</name>
    <value>true</value>
</property>

<property>
    <name>hive.cli.print.current.db</name>
    <value>true</value>
</property>

参考地址

• https://cwiki.apache.org/confluence/display/Hive/Home
• https://cwiki.apache.org/confluence/display/Hive/GettingStarted
• http://mirrors.163.com/.help/CentOS6-Base-163.repo