Configure the device plug-in
You must first define a resource name for the device plug-in to identify the resource
name. After you define a name, add the resource name in the yarn-site.xml
file.
-
Define the resource name in the
resource-types.xml
file for the pluggable device framework to identify the resource name the plug-in is handling.The following example defines the resource name asnvidia.com/gpu
:<property> <name>yarn.resource-types</name> <value>nvidia.com/gpu</value> </property>
-
Define the resource name handled by the plug-in in the
yarn-site.xml
file. The property value must be a full class name of the plug-in.For example:<property> <name>yarn.nodemanager.pluggable-device-framework.device-classes</name> <value>org.apache.hadoop.yarn.server.nodemanager.containermanager.resourceplugin.com.nvidia.NvidiaGPUPluginForRuntimeV2</value> </property>
From HDP 3.1.4 onwards, a new plug-in for NEC’s vector engine is available and you can enable the following configuration property to use it. Use comma separated multiple values in below configurations to use different hardware resources at the same time.
Add the following in the
resource-types.xml
file<property> <name>yarn.resource-types</name> <value>nec.com/ve</value> </property>
Add the following in the
yarn-site.xml
file<property> <name>yarn.nodemanager.pluggable-device-framework.device-classes</name> <value>org.apache.hadoop.yarn.server.nodemanager.containermanager.resourceplugin.com.nec.NECVEPlugin</value> </property>