在hive中,使用动态分区插入数据报dynamic.partitions.pernode. Maximum was set to: 100

报错如下:

The maximum number of dynamic partitions is controlled by hive.exec.max.dynamic.partitions and hive.exec.max.dynamic.partitions.pernode. Maximum was set to: 100

报错原因:

hive shell中默认的参数
hive.exec.max.dynamic.partitions.pernode
hive.exec.max.dynamic.partitions
hive.exec.max.dynamic.partitions.pernode.Maximum
hive.exec.max.created.files
是100
只允许一次性生成100个分区

查看参数命令:

set hive.exec.max.dynamic.partitions.pernode;
set hive.exec.max.dynamic.partitions.pernode.Maximum;
set hive.exec.max.dynamic.partitions;
set hive.exec.max.created.files;

解决办法:

在hive中添加配置,执行下列命令,重新运行就可以成功了

set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;
set hive.exec.max.dynamic.partitions.pernode=100000;
set hive.exec.max.dynamic.partitions.pernode.Maximum=100000;
set hive.exec.max.dynamic.partitions=100000;
set hive.exec.max.created.files=100000;

版权声明:本文为weixin_46086427原创文章,遵循CC 4.0 BY-SA版权协议,转载请附上原文出处链接和本声明。