安装azkaban-jobtype
1.下载azkaban-plugins
https://github.com/azkaban/azkaban-plugins
2.解压安装
unzip azkaban-plugins-3.0.0.zip
解压后目录:
cd plugins/jobtype/jobtypes
3.将jobtypes复制到azkaban-web-server-3.76.0/plugins/
4.修改配置
vim azkaban-exec-server-3.76.0/plugins/jobtypes/commonprivate.properties
## hadoop security manager setting common to all hadoop jobs
hadoop.security.manager.class=azkaban.security.HadoopSecurityManager_H_1_0## hadoop security related settings
# proxy.keytab.location=
# proxy.user=azkaban.should.proxy=false
obtain.binary.token=true
# obtain.namenode.token=true
# obtain.jobtracker.token=true# global classpath items for all jobs. e.g. hadoop-core jar, hadoop conf
#jobtype.global.classpath=${hive.home}/conf,${hive.home}/lib/*,${hadoop.home}/etc/hadoop,/usr/hdp/2.6.5.0-292/hadoop-hdfs/*,/usr/hdp/2.6.5.0-292/hadoop-hdfs/lib/*,/usr/hdp/2.6.5.0-292/hadoop-yarn/*,/usr/hdp/2.6.5.0-292/hadoop-yarn/lib/*,/usr/hdp/2.6.5.0-292/hadoop-mapreduce/*,/usr/hdp/2.6.5.0-292/hadoop-mapreduce/lib/*# global jvm args for all jobs. e.g. java.io.temp.dir, java.library.path
#jobtype.global.jvm.args=# hadoop
hadoop.home=/usr/hdp/2.6.5.0-292/hadoop/
hive.home=/usr/hdp/2.6.5.0-292/hive/
spark.home=/usr/hdp/2.6.5.0-292/spark/
#spark.home=# configs for jobtype security settings
jobtype.global.classpath=/usr/hdp/2.6.5.0-292/hadoop/etc/hadoop:/usr/hdp/2.6.5.0-292/hadoop-hdfs/*:/usr/hdp/2.6.5.0-292/hadoop-hdfs/lib/*:/usr/hdp/2.6.5.0-292/hadoop-yarn/*:/usr/hdp/2.6.5.0-292/hadoop-yarn/lib/*:/usr/hdp/2.6.5.0-292/hadoop-mapreduce/*:/usr/hdp/2.6.5.0-292/hadoop-mapreduce/lib/*
hadoop.classpath=/usr/hdp/2.6.5.0-292/hadoop/etc/hadoop:/usr/hdp/2.6.5.0-292/hadoop-hdfs/*:/usr/hdp/2.6.5.0-292/hadoop-hdfs/lib/*:/usr/hdp/2.6.5.0-292/hadoop-yarn/*:/usr/hdp/2.6.5.0-292/hadoop/etc-yarn/lib/*:/usr/hdp/2.6.5.0-292/hadoop-mapreduce/*:/usr/hdp/2.6.5.0-292/hadoop-mapreduce/lib/*
#execute.as.user=true
#azkaban.native.lib=
vim azkaban-exec-server-3.76.0/conf/azkaban.properties修改azkaban.properties
加上这个(如果没有)
# Azkaban plugin settings
azkaban.jobtype.plugin.dir=plugins/jobtypes
最后启动就可以了
报错:
2019/08/26 18:21:49.072 +0800 ERROR [StdOutErrRedirect] [Azkaban] at azkaban.jobtype.JobTypeManager.loadJobTypes(JobTypeManager.java:198)
2019/08/26 18:21:49.072 +0800 ERROR [StdOutErrRedirect] [Azkaban] ... 29 more
原因,没有配好配置文件
解决:第四步