问题如下:
17/10/26 16:33:46 WARN split.JobSplitWriter: Max block location exceeded for split: hdfs://youjun/input/file2:0+23 splitsize: 13 maxsize: 10
参考如下:
Issue: Max block location exceeded for split error when running hive
http://lucene.472066.n3.nabble.com/Issue-Max-block-location-exceeded-for-split-error-when-running-hive-td4090981.html
Max block location exceeded for split
http://tangjunliang.iteye.com/blog/2013744
解决方案:
set mapreduce.job.max.split.locations to at least the number of datanodes in your cluster, either in hive-site.xml or mapred-site.xml
也即在mapred-site.xml中
- <property>
- <name>mapreduce.job.max.split.locations</name>
- <value></value>
- </property>
版权声明:本文为Islotus原创文章,遵循CC 4.0 BY-SA版权协议,转载请附上原文出处链接和本声明。