Overriding the job-level instance type using the CLI [Technical Preview]

Spot instance settings are the default settings at Virtual Cluster (VC) level, however, to enhance flexibility, you can overwrite the defaults on a per-job basis.

This feature is applicable only if the VC-level instance type is a spot instance. This feature is not applicable if the VC-level instance type is on-demand. In such a case, setting the compute configuration at the job level results in an error.

To set the job level instance type override, you must configure the following CLIs:

CDP CLI Beta
- To install the CDP CLI Beta version, see Installing Beta CDP CLI.
- To configure the CDP CLI client, see CLI client setup.
CDE CLI
- To configure the CDE CLI client, see Configuring the CLI client.

You need the following variables for the commands:

CLUSTER-ID

To obtain the cluster ID:

Log in to the Cloudera Data Engineering UI and navigate to Administration in the left-navigation menu.
Select the respective service.
Click on the Service Details icon, and copy the CLUSTER ID.

VC-ID

To obtain the VC ID:

Log in to the Cloudera Data Engineering UI and navigate to Administration in the left-navigation menu.
Select the respective service and then the respective VC.
Click on the Virtual Cluster Details icon, and copy the VC-ID.

Enable the job-level instance type override for a new or an existing Cloudera Data Engineering VC using the CDP CLI Beta.
1. For creating a Cloudera Data Engineering VC, use the following command:
```
cdp de create-vc --cluster-id [***CLUSTER-ID***] --vc-id [***VC-ID***] --enable-compute-override
```
2. For an existing Cloudera Data Engineering VC, use the following command:
```
cdp de update-vc --cluster-id [***CLUSTER-ID***] --vc-id [***VC-ID***] --enable-compute-override
```
Override the selected instance type on Cloudera Data Engineering job level for a new or an existing job using the CDE CLI.
note
- For Spark, you can override the default virtual cluster settings to choose specific node types for Spark drivers and executors.
- For Airflow, you can override the default virtual cluster settings to choose specific node types for Airflow workers.
1. For creating a new job, run one of the following commands:
  - To use the instance type on job level that is set on the VC level, run the following command:
```
cde job create --compute-configuration "inherit" ...
```
  - To override the selected instance type on a Cloudera Data Engineering job level and set on-demand as the instance type, run the following command:
```
cde job create --compute-configuration "on-demand" ...
```
2. For updating an existing job, run one of the following commands:
  - To use the instance type on job level that is set on the VC level, run the following command:
```
cde job update --compute-configuration "inherit"
```
  - To override the selected instance type on a Cloudera Data Engineering job level and set on-demand as the instance type, run the following command:
```
cde job update --compute-configuration "on-demand"
```
Optional: Disable the job-level instance type override on a Cloudera Data Engineering VC using the following command in the CDP CLI.
note
If you disable the Job-level Instance Type Override feature, the compute configuration of the job reverts to the VC-level setting, and becomes read-only at the job level.
```
cdp de update-vc --cluster-id [***CLUSTER-ID***] --vc-id [***VC-ID***] --no-enable-compute-override
```