oozie

Oozie workflow中四个Action解析--Hive Action

Hive Action部署

创建action项目

oozie$:mkdir -p oozie-apps/hive/

拷贝hive-site.xml文件到项目文件夹下

oozie$:cp /opt/cdh3.5.6/hive-0.13.1-cdh5.3.6/conf/hive-site.xml oozie-apps/hive/

创建lib

oozie$:mkdir oozie-apps/hive/lib

导入mysqljar包

oozie$:cp /opt/cdh3.5.6/hive-0.13.1-cdh5.3.6/lib/mysql-connector-java-5.1.27-bin.jar oozie-apps/hive/lib

配置文件

• job.properties

nameNode=hdfs://hadoop-senior01.grc.com:8020
jobTracker=hadoop-senior01.grc.com:8032
queueName=default
oozieAppsRoot=user/grc/oozie-apps
oozieDataRoot=user/grc/oozie-datas

oozie.wf.application.path=${nameNode}/${oozieAppsRoot}/hive
oozie.use.system.libpath=true
outputDir=hive/output

• workflow.xml

<workflow-app xmlns="uri:oozie:workflow:0.5" name="hive-wf">
<start to="hive-node"/>

<action name="hive-node">
<hive xmlns="uri:oozie:hive-action:0.5">
<job-tracker>${jobTracker}</job-tracker>
    <name-node>${nameNode}</name-node>
<prepare>
<delete path="${nameNode}/${oozieDataRoot}/${outputDir}"/>
    </prepare>
    <!-- localtion file hive-site.xml-->
    			<job-xml>${nameNode}/${oozieAppsRoot}/hive/hive-site.xml</job-xml>
    <configuration>
    <property>
    <name>mapred.job.queue.name</name>
    <value>${queueName}</value>
</property>
</configuration>
<script>script.q</script>
<!--output-->
<param>OUTPUT=${nameNode}/${oozieDataRoot}/${outputDir}</param>
</hive>
<ok to="end"/>
<error to="fail"/>
</action>

<kill name="fail">
<message>Hive failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
</workflow-app>

Note:修改配置文件的版本号。

• script.q

insert overwrite directory'${OUTPUT}' 
select * from db_track.track_log limit 5;

创建HDFS文件系统上的目录

oozie$:/opt/cdh3.5.6/hadoop-2.5.0-cdh5.3.6/hdfs dfs -mkdiroozie-apps/hive/

输出目录

oozie$:/opt/cdh3.5.6/hadoop-2.5.0-cdh5.3.6/hdfs dfs -mkdir -p /user/grc/oozie-datas/hive/input

将项目文件上传至HDFS文件系统上

oozie$:/opt/cdh3.5.6/hadoop-2.5.0-cdh5.3.6/hdfs dfs -put oozie-apps/hive/*  oozie-apps/hive/

运行Hive Action

oozie$:bin/oozie job -config oozie-apps/hive/job.properties -run

hue中查看运行结果

参考地址：http://archive.cloudera.com/cdh5/cdh/5/oozie-4.0.0-cdh5.3.6/DG_HiveActionExtension.html