split.JobSplitWriter: Max block location exceeded for split

问题如下:

17/10/26 16:33:46 WARN split.JobSplitWriter: Max block location exceeded for split: hdfs://youjun/input/file2:0+23 splitsize: 13 maxsize: 10

参考如下:

Issue: Max block location exceeded for split error when running hive

http://lucene.472066.n3.nabble.com/Issue-Max-block-location-exceeded-for-split-error-when-running-hive-td4090981.html

Max block location exceeded for split

http://tangjunliang.iteye.com/blog/2013744

解决方案:
set mapreduce.job.max.split.locations to at least the number of datanodes in your cluster, either in hive-site.xml or mapred-site.xml
也即在mapred-site.xml中 

Java代码   收藏代码
  1. <property>  
  2.         <name>mapreduce.job.max.split.locations</name>  
  3.         <value></value>  
  4. </property>  
把这个值设置成你的datanodes数就行了。



版权声明:本文为Islotus原创文章,遵循CC 4.0 BY-SA版权协议,转载请附上原文出处链接和本声明。