亲测有效
版本
sqoop版本:1.4.7.3.1.4.0-315
hive版本:3.1.0.3.1.4.0-315
建表:
创建hive外部表:
CREATE external TABLE ods_base (
base_id string
) partitioned by (dt string)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '^'
STORED AS orc;
如果创建的是hive表内部表,会报错如下:
ERROR tool.ImportTool: Import failed: org.apache.hive.hcatalog.common.HCatException : 2016 : Error operation not supported : Store into a transactional table ods.ods_base from Pig/Mapreduce is not supported
改成外部表即可
sqoop导入脚本
#!/bin/bash
sqoop import \
--connect 'jdbc:mysql://test.mysql.com:3306/order' \
--username selectuser --password 'selectuser' \
--query "select base_id from base where create_date='2022-04-18' and 1=1 and \$CONDITIONS" \
--driver com.mysql.jdbc.Driver \
--hcatalog-database ods \
--hcatalog-table ods_base \
--hcatalog-partition-keys dt \
--hcatalog-partition-values '2022-04-18' \
-m 1