Managing Data Operating System
Also available as:
PDF
loading table of contents...

Configure the device plug-in

You must first define a resource name for the device plug-in to identify the resource name. After you define a name, add the resource name in the yarn-site.xml file.

  1. Define the resource name in the resource-types.xml file for the pluggable device framework to identify the resource name the plug-in is handling.
    The following example defines the resource name as nvidia.com/gpu:
    <property>
      <name>yarn.resource-types</name>
      <value>nvidia.com/gpu</value>
    </property>
  2. Define the resource name handled by the plug-in in the yarn-site.xml file. The property value must be a full class name of the plug-in.
    For example:
    <property>
      <name>yarn.nodemanager.pluggable-device-framework.device-classes</name>
     <value>org.apache.hadoop.yarn.server.nodemanager.containermanager.resourceplugin.com.nvidia.NvidiaGPUPluginForRuntimeV2</value>
    </property>

    From HDP 3.1.4 onwards, a new plug-in for NEC’s vector engine is available and you can enable the following configuration property to use it. Use comma separated multiple values in below configurations to use different hardware resources at the same time.

    Add the following in the resource-types.xml file

    <property>
      <name>yarn.resource-types</name>
       <value>nec.com/ve</value>
    </property>

    Add the following in the yarn-site.xml file

    <property>
      <name>yarn.nodemanager.pluggable-device-framework.device-classes</name>
       <value>org.apache.hadoop.yarn.server.nodemanager.containermanager.resourceplugin.com.nec.NECVEPlugin</value>
    </property>