You can create a job in Cloudera Data Engineering (CDE) using the
CDE jobs
API endpoint.
-
Determine the API URL for the virtual cluster you want to access
using the API:
-
In the Cloudera Data Platform (CDP) console, click the
Data Engineering tile. The CDE Home
page displays.
-
In the Virtual Clusters section, navigate to the virtual
cluster for which you want to interact with, and click Cluster
Details.
-
Click JOBS API URL to copy the URL.
For example:
https://pmjkrgn5.cde-czlmkz4y.na-01.xvp2-7p8o.cloudera.site/dex/api/v1
-
Submit the job creation request using the API to the
/jobs
endpoint.
The JSON payload to create a job is structured as
follows:
{
"name": "demoJob",
"spark": {
"className": "com.example.demoJobMainClass",
"file": "local:/path/to/demoJobJar"
},
"type": "spark"
}
curl -H "Authorization: Bearer ${CDE_TOKEN}" <jobs_api_url>/jobs \
-H "Content-Type: application/json" \
-X POST -d "{\"name\":\"demoJob\",\"spark\":{\"className\":\"com.example.demoJobMainClass\",\"file\":\"local:/path/to/demoJobJar\"},\"type\":\"spark\"}"
- Verify the job was created. You can view job details using the
/jobs/<jobName>
endpoint:
curl -H "Authorization: Bearer ${CDE_TOKEN}" -H "Content-Type: application/json" -X GET "https://pmjkrgn5.cde-czlmkz4y.na-01.xvp2-7p8o.cloudera.site/dex/api/v1/jobs/demoJob"