hadoop yarn队列资源配置

yarn队列资源配置详解

yarn现有资源查看

http://master:8088 //默认端口 8088

yarn队列资源详细设置

修改配置文件capacity-scheduler.xml

//指定队列名
<property>
    <name>yarn.scheduler.capacity.root.queues</name>
    <value>jobs,analyze,dev</value>
</property>

//分配每个队列的资源占用
<property>
  <name>yarn.scheduler.capacity.root.jobs.capacity</name>
  <value>60</value>
</property>

<property>
   <name>yarn.scheduler.capacity.root.analyze.capacity</name>
   <value>30</value>
</property>

<property>
    <name>yarn.scheduler.capacity.root.dev.capacity</name>
    <value>10</value>
</property>

jobs队列详细配置

<property>
 <name>yarn.scheduler.capacity.root.jobs.user-limit-factor</name>
 <value>1</value>
</property>
<property>
 <name>yarn.scheduler.capacity.root.jobs.maximum-capacity</name>
 <value>80</value>
</property>
<property>
 <name>yarn.scheduler.capacity.root.jobs.state</name>
 <value>RUNNING</value>
</property>
<property>
 <name>yarn.scheduler.capacity.root.jobs.acl_submit_applications</name>
 <value>*</value>
</property>
<property>
 <name>yarn.scheduler.capacity.root.jobs.acl_administer_queue</name>
 <value>*</value>
</property>

analyze队列详细配置

<property>
 <name>yarn.scheduler.capacity.root.analyze.user-limit-factor</name>
 <value>1</value>
</property>
<property>
 <name>yarn.scheduler.capacity.root.analyze.maximum-capacity</name>
 <value>80</value>
</property>
<property>
 <name>yarn.scheduler.capacity.root.analyze.state</name>
 <value>RUNNING</value>
</property>
<property>
 <name>yarn.scheduler.capacity.root.analyze.acl_submit_applications</name>
 <value>*</value>
</property>
<property>
 <name>yarn.scheduler.capacity.root.analyze.acl_administer_queue</name>
 <value>*</value>
</property>

dev队列详细配置

<property>
 <name>yarn.scheduler.capacity.root.dev.user-limit-factor</name>
 <value>1</value>
</property>
<property>
 <name>yarn.scheduler.capacity.root.dev.maximum-capacity</name>
 <value>80</value>
</property>
<property>
 <name>yarn.scheduler.capacity.root.dev.state</name>
 <value>RUNNING</value>
</property>
<property>
 <name>yarn.scheduler.capacity.root.dev.acl_submit_applications</name>
 <value>*</value>
</property>
<property>
 <name>yarn.scheduler.capacity.root.dev.acl_administer_queue</name>
 <value>*</value>
</property>

配置完成后重新检查资源情况：