Azkaban运行MapReduce报错:is running beyond virtual memory limits


错误描述


  今天使用Azkaban运行Hadoop自带的单词计数jar报以下错误:
[pid=19143,containerID=container_1515150928558_0003_01_000002] is running beyond virtual memory limits. Current usage: 105.8 MB of 1 GB physical memory used; 2.8 GB of 2.1 GB virtual memory used. Killing container.node

  通过上网查询资料,终于获得有效解决。oop

错误缘由


  报该错误的缘由是由于运行所须要的内存超出虚拟内存限制,container被迫杀死致使任务不能正常进行。spa

解决办法


  切换到/root/hadoop-2.7.2/etc/hadoop修改yarn-site.xml添加如下内容:3d

<!--当运行时所需内存超过虚拟内存限制是否强制关闭container-->
<property>
    <name>yarn.nodemanager.vmem-check-enabled</name>
    <value>false</value>
    <description>Whether virtual memory limits will be enforced for containers</description>
</property>
<!--设置容器的内存限制时虚拟内存与物理内存之间的比率-->
<property>
    <name>yarn.nodemanager.vmem-pmem-ratio</name>
    <value>4</value>
    <description>Ratio between virtual memory to physical memory when setting memory limits for containers</description>
</property>


结果code

相关文章
相关标签/搜索