CDE Jobs API

A set of APIs to run and manage jobs on CDE Virtual Clusters
More information: https://docs.cloudera.com/data-engineering/cloud/index.html
Contact Info: Unavailable
Version: 1.5.0
BasePath:/dex/api/v1
All rights reserved
http://apache.org/licenses/LICENSE-2.0.html

Access

  1. APIKey KeyParamName:Authorization KeyInQuery:false KeyInHeader:true

Methods

[ Jump to Models ]

Table of Contents

Admin

AdminAirflow

Credentials

JobRuns

Jobs

JobsSchedule

Meta

Resources

Admin

Up
get /admin/export
Exports jobs and resources (export)
Returns an archive continaing all jobs and resources in a virtual cluster. Optional Filters can limit what is exported.

Query parameters

exportjobs (optional)
Query Parameter — export all jobs in the virtual cluster default: true
jobfilter (optional)
Query Parameter — If exportjobs is true, only export jobs that the meet the passed filter syntax 'fieldname[operator]argument'. 'fieldname' is the name of a job API field. 'operator' is one of ('eq', 'noteq', 'lte', 'lt', 'gte', 'gt', 'in', 'notin', 'like', 'rlike'). For example, 'created[gte]2020-01-01'. Multiple filters are ANDed.
exportjobresources (optional)
Query Parameter — also export resources needed for jobs - selected resources cannot be filtered out by resourcefilter parameter default: false
exportresources (optional)
Query Parameter — export all resource in the virtual cluster default: false
resourcefilter (optional)
Query Parameter — If exportresources is true, only export resources that meet the passed filter syntax 'fieldname[operator]argument'. 'fieldname' is the name of a resource API field. 'operator' is one of ('eq', 'noteq', 'lte', 'lt', 'gte', 'gt', 'in', 'notin', 'like', 'rlike'). For example, 'name[noteq]my-resource'. Multiple filters are ANDed.

Return type

String

Example data

Content-Type:

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

200

OK String

500

Internal Server Error util.Response

Up
post /admin/import
Restore one or more jobs from an archive (import)
Restores jobs from a passed archive into the current virtual cluster

Consumes

This API call consumes the following media types via the Content-Type request header:

Form parameters

file (required)
Form Parameter — The archive file to restore
duplicatehandling (optional)
Form Parameter — One of error, merge, or rename default: error

Return type

array[common.ArchiveListing]

Example data

Content-Type: application/json
[ {
  "cdeVersion" : "cdeVersion",
  "entityType" : "entityType",
  "envName" : "envName",
  "adjustedName" : "adjustedName",
  "appID" : "appID",
  "name" : "name",
  "clusterID" : "clusterID",
  "archiveDirectoryPath" : "archiveDirectoryPath",
  "backupDate" : "backupDate",
  "user" : "user"
}, {
  "cdeVersion" : "cdeVersion",
  "entityType" : "entityType",
  "envName" : "envName",
  "adjustedName" : "adjustedName",
  "appID" : "appID",
  "name" : "name",
  "clusterID" : "clusterID",
  "archiveDirectoryPath" : "archiveDirectoryPath",
  "backupDate" : "backupDate",
  "user" : "user"
} ]

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

200

OK

500

Internal Server Error util.Response

Up
post /admin/list-archive
Lists jobs in an archive (listArchive)
list jobs from a passed archive

Consumes

This API call consumes the following media types via the Content-Type request header:

Form parameters

file (required)
Form Parameter — The archive file to list jobs from

Return type

array[common.ArchiveListing]

Example data

Content-Type: application/json
[ {
  "cdeVersion" : "cdeVersion",
  "entityType" : "entityType",
  "envName" : "envName",
  "adjustedName" : "adjustedName",
  "appID" : "appID",
  "name" : "name",
  "clusterID" : "clusterID",
  "archiveDirectoryPath" : "archiveDirectoryPath",
  "backupDate" : "backupDate",
  "user" : "user"
}, {
  "cdeVersion" : "cdeVersion",
  "entityType" : "entityType",
  "envName" : "envName",
  "adjustedName" : "adjustedName",
  "appID" : "appID",
  "name" : "name",
  "clusterID" : "clusterID",
  "archiveDirectoryPath" : "archiveDirectoryPath",
  "backupDate" : "backupDate",
  "user" : "user"
} ]

Responses

200

OK

500

Internal Server Error util.Response

AdminAirflow

Up
delete /admin/airflow/dags/{dag-id}
Delete an unassociated Airflow DAG (deleteAirflowDag)
Delete Airflow DAGs that are not associated with any jobs

Path parameters

dag-id (required)
Path Parameter — DAG ID

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

204

404

Not Found util.Response

500

Internal Server Error util.Response

Credentials

Up
post /credentials
Create a credential (Experimental) (createCredential)
Creates a managed credential stored as a Kubernetes secret. (Experimental)

Consumes

This API call consumes the following media types via the Content-Type request header:

Request body

credential common.CredentialCreateRequest (required)
Body Parameter — Credential definition

Responses

201

400

Bad Request util.Response

404

Not Found util.Response

500

Internal Server Error util.Response

Up
delete /credentials/{name}
Delete a credential (Experimental) (deleteCredential)
Delete named credential (Experimental)

Path parameters

name (required)
Path Parameter — Credential name

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

204

404

Not Found util.Response

500

Internal Server Error util.Response

Up
get /credentials/{name}
Describe a credential (Experimental) (getCredential)
Describe specific credential (Experimental)

Path parameters

name (required)
Path Parameter — Credential name

Return type

common.Credential

Example data

Content-Type: application/json
{
  "metadata" : {
    "key" : "metadata"
  },
  "hidden" : true,
  "created" : "created",
  "name" : "name",
  "description" : "description",
  "modified" : "modified",
  "type" : "type"
}

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

200

OK common.Credential

404

Not Found common.Credential

500

Internal Server Error util.Response

Up
get /credentials
List all credential (Experimental) (listCredentials)
List all credential (Experimental)

Query parameters

filter (optional)
Query Parameter — Filter the list by the syntax 'fieldname[operator]argument'. 'fieldname' is the name of a credential API field. 'operator' is one of ('eq', 'noteq', 'lte', 'lt', 'gte', 'gt', 'in', 'notin', 'like', 'rlike'). For example, 'created[gte]2020-01-01'. Multiple filters are ANDed.
limit (optional)
Query Parameter — The maximum number of credentials to return, up to 100 default: 20
offset (optional)
Query Parameter — The number of credentials to skip before starting list default: 0
orderby (optional)
Query Parameter — The credential API field to order by default: name
orderasc (optional)
Query Parameter — Whether an ordering is ascending default: true

Return type

common.CredentialListResponse

Example data

Content-Type: application/json
{
  "credentials" : [ {
    "metadata" : {
      "key" : "metadata"
    },
    "hidden" : true,
    "created" : "created",
    "name" : "name",
    "description" : "description",
    "modified" : "modified",
    "type" : "type"
  }, {
    "metadata" : {
      "key" : "metadata"
    },
    "hidden" : true,
    "created" : "created",
    "name" : "name",
    "description" : "description",
    "modified" : "modified",
    "type" : "type"
  } ],
  "meta" : {
    "offset" : 1,
    "count" : 0,
    "limit" : 6,
    "hasNext" : true
  }
}

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

200

OK common.CredentialListResponse

500

Internal Server Error util.Response

Up
patch /credentials/{name}
Update a credential (Experimental) (updateCredential)
Update named credential (Experimental)

Path parameters

name (required)
Path Parameter — Credential name

Request body

credential common.CredentialUpdateRequest (required)
Body Parameter — Credential definition

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

204

404

Not Found util.Response

500

Internal Server Error util.Response

JobRuns

Up
get /job-runs/{id}
Describe a job run (getJobRun)
Describe the job run configuration and status

Path parameters

id (required)
Path Parameter — Job run ID

Return type

common.Run

Example data

Content-Type: application/json
{
  "hidden" : true,
  "runtimeImageResourceName" : "runtimeImageResourceName",
  "spark" : {
    "sparkAppID" : "sparkAppID",
    "batchID" : 6,
    "sparkAppURL" : "sparkAppURL",
    "spec" : {
      "pythonEnvResourceName" : "pythonEnvResourceName",
      "driverMemory" : "driverMemory",
      "executorMemory" : "executorMemory",
      "driverCores" : 1,
      "executorCores" : 5,
      "jars" : [ "jars", "jars" ],
      "className" : "className",
      "conf" : {
        "key" : "conf"
      },
      "pyFiles" : [ "pyFiles", "pyFiles" ],
      "args" : [ "args", "args" ],
      "numExecutors" : 5,
      "file" : "file",
      "logLevel" : "logLevel",
      "name" : "name",
      "proxyUser" : "proxyUser",
      "files" : [ "files", "files" ]
    }
  },
  "airflow" : {
    "dagRunID" : "dagRunID",
    "dagFile" : "dagFile",
    "executionDate" : "executionDate",
    "dagID" : "dagID",
    "config" : {
      "key" : "config"
    },
    "startDate" : "startDate"
  },
  "ended" : "ended",
  "mounts" : [ {
    "dirPrefix" : "dirPrefix",
    "resourceName" : "resourceName"
  }, {
    "dirPrefix" : "dirPrefix",
    "resourceName" : "resourceName"
  } ],
  "started" : "started",
  "id" : 0,
  "job" : "job",
  "type" : "type",
  "user" : "user",
  "status" : "status"
}

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

200

OK common.Run

400

Bad Request util.Response

404

Not Found util.Response

500

Internal Server Error util.Response

Up
get /job-runs/{id}/log-types
List a job run's log types (getJobRunLogTypes)

Path parameters

id (required)
Path Parameter — Job run ID

Return type

array[common.RunLogType]

Example data

Content-Type: application/json
[ {
  "entityLabel" : "entityLabel",
  "streamLabel" : "streamLabel",
  "type" : "type",
  "entityDefault" : true
}, {
  "entityLabel" : "entityLabel",
  "streamLabel" : "streamLabel",
  "type" : "type",
  "entityDefault" : true
} ]

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

200

OK

400

Bad Request util.Response

404

Not Found util.Response

500

Internal Server Error util.Response

Up
get /job-runs/{id}/logs
Get logs for a job run (getJobRunLogs)

Path parameters

id (required)
Path Parameter — Job run ID

Query parameters

type (optional)
Query Parameter — Log type within job run. Use 'all' for zip file of all log types
follow (optional)
Query Parameter — Follow log if in progress
tailLines (optional)
Query Parameter — Number of lines to tail
download (optional)
Query Parameter — Advise browser to download log

Return type

array[String]

Example data

Content-Type:

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

200

OK

400

Bad Request util.Response

404

Not Found util.Response

500

Internal Server Error util.Response

Up
post /job-runs/{id}/kill
Kill a job run (killJobRun)
Kill the specified job run

Path parameters

id (required)
Path Parameter — Job run ID

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

200

400

Bad Request util.Response

404

Not Found util.Response

500

Internal Server Error util.Response

Up
get /job-runs
List job runs (listJobRuns)
List job runs, optionally filtering by job name and/or run status

Query parameters

filter (optional)
Query Parameter — Filter the list by the syntax 'fieldname[operator]argument'. 'fieldname' is the name of a job run API field. 'operator' is one of ('eq', 'noteq', 'lte', 'lt', 'gte', 'gt', 'in', 'notin', 'like', 'rlike'). For example, 'name[noteq]my-resource'. Multiple filters are ANDed.
limit (optional)
Query Parameter — The maximum number of job runs to return, up to 100 default: 20
offset (optional)
Query Parameter — The number of job runs to skip before starting list default: 0
orderby (optional)
Query Parameter — The job run API field to order by default: ID
orderasc (optional)
Query Parameter — Whether an ordering is ascending default: true
latestby (optional)
Query Parameter — Latest job run only for each unique job or user

Return type

common.RunListResponse

Example data

Content-Type: application/json
{
  "meta" : {
    "offset" : 1,
    "count" : 0,
    "limit" : 6,
    "hasNext" : true
  },
  "runs" : [ {
    "hidden" : true,
    "runtimeImageResourceName" : "runtimeImageResourceName",
    "spark" : {
      "sparkAppID" : "sparkAppID",
      "batchID" : 6,
      "sparkAppURL" : "sparkAppURL",
      "spec" : {
        "pythonEnvResourceName" : "pythonEnvResourceName",
        "driverMemory" : "driverMemory",
        "executorMemory" : "executorMemory",
        "driverCores" : 1,
        "executorCores" : 5,
        "jars" : [ "jars", "jars" ],
        "className" : "className",
        "conf" : {
          "key" : "conf"
        },
        "pyFiles" : [ "pyFiles", "pyFiles" ],
        "args" : [ "args", "args" ],
        "numExecutors" : 5,
        "file" : "file",
        "logLevel" : "logLevel",
        "name" : "name",
        "proxyUser" : "proxyUser",
        "files" : [ "files", "files" ]
      }
    },
    "airflow" : {
      "dagRunID" : "dagRunID",
      "dagFile" : "dagFile",
      "executionDate" : "executionDate",
      "dagID" : "dagID",
      "config" : {
        "key" : "config"
      },
      "startDate" : "startDate"
    },
    "ended" : "ended",
    "mounts" : [ {
      "dirPrefix" : "dirPrefix",
      "resourceName" : "resourceName"
    }, {
      "dirPrefix" : "dirPrefix",
      "resourceName" : "resourceName"
    } ],
    "started" : "started",
    "id" : 0,
    "job" : "job",
    "type" : "type",
    "user" : "user",
    "status" : "status"
  }, {
    "hidden" : true,
    "runtimeImageResourceName" : "runtimeImageResourceName",
    "spark" : {
      "sparkAppID" : "sparkAppID",
      "batchID" : 6,
      "sparkAppURL" : "sparkAppURL",
      "spec" : {
        "pythonEnvResourceName" : "pythonEnvResourceName",
        "driverMemory" : "driverMemory",
        "executorMemory" : "executorMemory",
        "driverCores" : 1,
        "executorCores" : 5,
        "jars" : [ "jars", "jars" ],
        "className" : "className",
        "conf" : {
          "key" : "conf"
        },
        "pyFiles" : [ "pyFiles", "pyFiles" ],
        "args" : [ "args", "args" ],
        "numExecutors" : 5,
        "file" : "file",
        "logLevel" : "logLevel",
        "name" : "name",
        "proxyUser" : "proxyUser",
        "files" : [ "files", "files" ]
      }
    },
    "airflow" : {
      "dagRunID" : "dagRunID",
      "dagFile" : "dagFile",
      "executionDate" : "executionDate",
      "dagID" : "dagID",
      "config" : {
        "key" : "config"
      },
      "startDate" : "startDate"
    },
    "ended" : "ended",
    "mounts" : [ {
      "dirPrefix" : "dirPrefix",
      "resourceName" : "resourceName"
    }, {
      "dirPrefix" : "dirPrefix",
      "resourceName" : "resourceName"
    } ],
    "started" : "started",
    "id" : 0,
    "job" : "job",
    "type" : "type",
    "user" : "user",
    "status" : "status"
  } ]
}

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

200

OK common.RunListResponse

500

Internal Server Error util.Response

Jobs

Up
post /jobs
Create a job (createJob)
Creates a new job definition

Consumes

This API call consumes the following media types via the Content-Type request header:

Request body

Body Parameter — Job definition

Responses

201

400

Bad Request util.Response

404

Not Found util.Response

500

Internal Server Error util.Response

Up
delete /jobs/{name}
Delete a job (deleteJob)
Delete named job

Path parameters

name (required)
Path Parameter — Job name

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

204

404

Not Found util.Response

500

Internal Server Error util.Response

Up
get /jobs/{name}
Describe a job (getJob)
Describe job and its configuration

Path parameters

name (required)
Path Parameter — Job name

Query parameters

latestjob (optional)
Query Parameter — Include latest job information, if any

Return type

common.Job

Example data

Content-Type: application/json
{
  "lastUsed" : "lastUsed",
  "schedule" : {
    "cronExpression" : "cronExpression",
    "dependsOnPast" : true,
    "paused" : true,
    "catchup" : true,
    "pausedUponCreation" : true,
    "start" : "start",
    "end" : "end",
    "nextExecution" : "nextExecution",
    "user" : "user",
    "enabled" : true
  },
  "hidden" : true,
  "runtimeImageResourceName" : "runtimeImageResourceName",
  "spark" : {
    "pythonEnvResourceName" : "pythonEnvResourceName",
    "driverMemory" : "driverMemory",
    "executorMemory" : "executorMemory",
    "driverCores" : 1,
    "executorCores" : 5,
    "jars" : [ "jars", "jars" ],
    "className" : "className",
    "conf" : {
      "key" : "conf"
    },
    "pyFiles" : [ "pyFiles", "pyFiles" ],
    "args" : [ "args", "args" ],
    "numExecutors" : 5,
    "file" : "file",
    "logLevel" : "logLevel",
    "name" : "name",
    "proxyUser" : "proxyUser",
    "files" : [ "files", "files" ]
  },
  "created" : "created",
  "airflow" : {
    "dagFile" : "dagFile",
    "dagID" : "dagID",
    "config" : {
      "key" : "config"
    }
  },
  "name" : "name",
  "modified" : "modified",
  "mounts" : [ {
    "dirPrefix" : "dirPrefix",
    "resourceName" : "resourceName"
  }, {
    "dirPrefix" : "dirPrefix",
    "resourceName" : "resourceName"
  } ],
  "retentionPolicy" : "retentionPolicy",
  "type" : "type"
}

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

200

OK common.Job

404

Not Found util.Response

500

Internal Server Error util.Response

Up
get /jobs
List all jobs (listJobs)
List all jobs and their configurations

Query parameters

latestjob (optional)
Query Parameter — Include latest job information, if any default: false
filter (optional)
Query Parameter — Filter the list by the syntax 'fieldname[operator]argument'. 'fieldname' is the name of a job API field. 'operator' is one of ('eq', 'noteq', 'lte', 'lt', 'gte', 'gt', 'in', 'notin', 'like', 'rlike'). For example, 'created[gte]2020-01-01'. Multiple filters are ANDed.
limit (optional)
Query Parameter — The maximum number of jobs to return, up to 100 default: 20
offset (optional)
Query Parameter — The number of jobs to skip before starting list default: 0
orderby (optional)
Query Parameter — The job API field to order by default: name
orderasc (optional)
Query Parameter — Whether an ordering is ascending default: true

Return type

common.JobListResponse

Example data

Content-Type: application/json
{
  "meta" : {
    "offset" : 1,
    "count" : 0,
    "limit" : 6,
    "hasNext" : true
  },
  "jobs" : [ {
    "hidden" : true,
    "runtimeImageResourceName" : "runtimeImageResourceName",
    "created" : "created",
    "airflow" : {
      "dagFile" : "dagFile",
      "dagID" : "dagID",
      "config" : {
        "key" : "config"
      }
    },
    "mounts" : [ {
      "dirPrefix" : "dirPrefix",
      "resourceName" : "resourceName"
    }, {
      "dirPrefix" : "dirPrefix",
      "resourceName" : "resourceName"
    } ],
    "type" : "type",
    "lastUsed" : "lastUsed",
    "schedule" : {
      "cronExpression" : "cronExpression",
      "dependsOnPast" : true,
      "paused" : true,
      "catchup" : true,
      "pausedUponCreation" : true,
      "start" : "start",
      "end" : "end",
      "nextExecution" : "nextExecution",
      "user" : "user",
      "enabled" : true
    },
    "spark" : {
      "pythonEnvResourceName" : "pythonEnvResourceName",
      "driverMemory" : "driverMemory",
      "executorMemory" : "executorMemory",
      "driverCores" : 1,
      "executorCores" : 5,
      "jars" : [ "jars", "jars" ],
      "className" : "className",
      "conf" : {
        "key" : "conf"
      },
      "pyFiles" : [ "pyFiles", "pyFiles" ],
      "args" : [ "args", "args" ],
      "numExecutors" : 5,
      "file" : "file",
      "logLevel" : "logLevel",
      "name" : "name",
      "proxyUser" : "proxyUser",
      "files" : [ "files", "files" ]
    },
    "name" : "name",
    "modified" : "modified",
    "retentionPolicy" : "retentionPolicy",
    "latestRunInfo" : {
      "hidden" : true,
      "runtimeImageResourceName" : "runtimeImageResourceName",
      "spark" : {
        "sparkAppID" : "sparkAppID",
        "batchID" : 6,
        "sparkAppURL" : "sparkAppURL",
        "spec" : {
          "pythonEnvResourceName" : "pythonEnvResourceName",
          "driverMemory" : "driverMemory",
          "executorMemory" : "executorMemory",
          "driverCores" : 1,
          "executorCores" : 5,
          "jars" : [ "jars", "jars" ],
          "className" : "className",
          "conf" : {
            "key" : "conf"
          },
          "pyFiles" : [ "pyFiles", "pyFiles" ],
          "args" : [ "args", "args" ],
          "numExecutors" : 5,
          "file" : "file",
          "logLevel" : "logLevel",
          "name" : "name",
          "proxyUser" : "proxyUser",
          "files" : [ "files", "files" ]
        }
      },
      "airflow" : {
        "dagRunID" : "dagRunID",
        "dagFile" : "dagFile",
        "executionDate" : "executionDate",
        "dagID" : "dagID",
        "config" : {
          "key" : "config"
        },
        "startDate" : "startDate"
      },
      "ended" : "ended",
      "mounts" : [ {
        "dirPrefix" : "dirPrefix",
        "resourceName" : "resourceName"
      }, {
        "dirPrefix" : "dirPrefix",
        "resourceName" : "resourceName"
      } ],
      "started" : "started",
      "id" : 0,
      "job" : "job",
      "type" : "type",
      "user" : "user",
      "status" : "status"
    }
  }, {
    "hidden" : true,
    "runtimeImageResourceName" : "runtimeImageResourceName",
    "created" : "created",
    "airflow" : {
      "dagFile" : "dagFile",
      "dagID" : "dagID",
      "config" : {
        "key" : "config"
      }
    },
    "mounts" : [ {
      "dirPrefix" : "dirPrefix",
      "resourceName" : "resourceName"
    }, {
      "dirPrefix" : "dirPrefix",
      "resourceName" : "resourceName"
    } ],
    "type" : "type",
    "lastUsed" : "lastUsed",
    "schedule" : {
      "cronExpression" : "cronExpression",
      "dependsOnPast" : true,
      "paused" : true,
      "catchup" : true,
      "pausedUponCreation" : true,
      "start" : "start",
      "end" : "end",
      "nextExecution" : "nextExecution",
      "user" : "user",
      "enabled" : true
    },
    "spark" : {
      "pythonEnvResourceName" : "pythonEnvResourceName",
      "driverMemory" : "driverMemory",
      "executorMemory" : "executorMemory",
      "driverCores" : 1,
      "executorCores" : 5,
      "jars" : [ "jars", "jars" ],
      "className" : "className",
      "conf" : {
        "key" : "conf"
      },
      "pyFiles" : [ "pyFiles", "pyFiles" ],
      "args" : [ "args", "args" ],
      "numExecutors" : 5,
      "file" : "file",
      "logLevel" : "logLevel",
      "name" : "name",
      "proxyUser" : "proxyUser",
      "files" : [ "files", "files" ]
    },
    "name" : "name",
    "modified" : "modified",
    "retentionPolicy" : "retentionPolicy",
    "latestRunInfo" : {
      "hidden" : true,
      "runtimeImageResourceName" : "runtimeImageResourceName",
      "spark" : {
        "sparkAppID" : "sparkAppID",
        "batchID" : 6,
        "sparkAppURL" : "sparkAppURL",
        "spec" : {
          "pythonEnvResourceName" : "pythonEnvResourceName",
          "driverMemory" : "driverMemory",
          "executorMemory" : "executorMemory",
          "driverCores" : 1,
          "executorCores" : 5,
          "jars" : [ "jars", "jars" ],
          "className" : "className",
          "conf" : {
            "key" : "conf"
          },
          "pyFiles" : [ "pyFiles", "pyFiles" ],
          "args" : [ "args", "args" ],
          "numExecutors" : 5,
          "file" : "file",
          "logLevel" : "logLevel",
          "name" : "name",
          "proxyUser" : "proxyUser",
          "files" : [ "files", "files" ]
        }
      },
      "airflow" : {
        "dagRunID" : "dagRunID",
        "dagFile" : "dagFile",
        "executionDate" : "executionDate",
        "dagID" : "dagID",
        "config" : {
          "key" : "config"
        },
        "startDate" : "startDate"
      },
      "ended" : "ended",
      "mounts" : [ {
        "dirPrefix" : "dirPrefix",
        "resourceName" : "resourceName"
      }, {
        "dirPrefix" : "dirPrefix",
        "resourceName" : "resourceName"
      } ],
      "started" : "started",
      "id" : 0,
      "job" : "job",
      "type" : "type",
      "user" : "user",
      "status" : "status"
    }
  } ]
}

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

200

OK common.JobListResponse

500

Internal Server Error util.Response

Up
post /jobs/{name}/run
Run a job (runJob)
Run an instance of the specified job

Path parameters

name (required)
Path Parameter — Job name

Consumes

This API call consumes the following media types via the Content-Type request header:

Request body

spec common.RunRequest (required)
Body Parameter — Run definition

Return type

common.RunJobResponse

Example data

Content-Type: application/json
{
  "id" : 0
}

Responses

201

Created common.RunJobResponse

400

Bad Request util.Response

404

Not Found util.Response

500

Internal Server Error util.Response

Up
patch /jobs/{name}
Update a job (updateJob)
Update named job with new configuration

Path parameters

name (required)
Path Parameter — Job name

Request body

Body Parameter — Job definition

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

204

404

Not Found util.Response

500

Internal Server Error util.Response

JobsSchedule

Up
post /jobs/{name}/schedule/clear
Clear a job schedule (clear)
Clear schedule for named job

Path parameters

name (required)
Path Parameter — Job name

Request body

Body Parameter — Schedule clear definition

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

204

404

Not Found util.Response

500

Internal Server Error util.Response

Up
post /jobs/{name}/schedule/mark-success
Mark a job schedule as successful (markSuccess)
Mark schedule success for named job

Path parameters

name (required)
Path Parameter — Job name

Request body

Body Parameter — Schedule mark success definition

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

204

404

Not Found util.Response

500

Internal Server Error util.Response

Up
post /jobs/{name}/schedule/pause
Pause a job schedule (pause)
Pause schedule for named job

Path parameters

name (required)
Path Parameter — Job name

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

204

404

Not Found util.Response

500

Internal Server Error util.Response

Up
post /jobs/{name}/schedule/unpause
Unpause a job schedule (unpause)
Unpause schedule for named job

Path parameters

name (required)
Path Parameter — Job name

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

204

404

Not Found util.Response

500

Internal Server Error util.Response

Meta

Up
get /info
Information about the instance (info)
Configuration information and useful parameters for the instance

Return type

array[common.Info]

Example data

Content-Type: application/json
[ { }, { } ]

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

200

OK

500

Internal Server Error util.Response

Up
get /user
Information about the authenticated user (user)
Information about the authenticated user or empty if no authentication

Return type

common.UserDetail

Example data

Content-Type: application/json
{
  "displayName" : "displayName",
  "username" : "username"
}

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

200

OK common.UserDetail

400

Bad Request util.Response

500

Internal Server Error util.Response

Resources

Up
post /resources
Create a resource (createResource)
Creates a new resource with the supplied name and type supported resource types: files, python-env experimental resource types: custom-runtime-image

Consumes

This API call consumes the following media types via the Content-Type request header:

Request body

spec common.ResourceRequest (required)
Body Parameter — Resource definition

Responses

201

400

Bad Request util.Response

409

Conflict util.Response

500

Internal Server Error util.Response

Up
delete /resources/{name}
Delete a resource (deleteResource)
Delete resource definition and all associated content

Path parameters

name (required)
Path Parameter — Resource name

Responses

204

400

Bad Request util.Response

404

Not Found util.Response

500

Internal Server Error util.Response

Up
delete /resources/{name}/{path}
Delete a resource file (deleteResourceFile)
Delete a file in the resource at the path specified

Path parameters

name (required)
Path Parameter — Resource name
path (required)
Path Parameter — File resource path

Responses

204

400

Bad Request util.Response

404

Not Found util.Response

500

Internal Server Error util.Response

Up
get /resources/{name}
Describe a resource (getResource)
Get resource configuration and content

Path parameters

name (required)
Path Parameter — Resource name

Query parameters

includeFiles (optional)
Query Parameter — Whether to include the files of the resource default: true

Return type

common.Resource

Example data

Content-Type: application/json
{
  "hidden" : true,
  "signature" : "signature",
  "created" : "created",
  "pythonEnv" : {
    "pythonVersion" : "pythonVersion",
    "pyPiMirror" : "pyPiMirror"
  },
  "packages" : [ {
    "created" : "created",
    "name" : "name",
    "type" : "type",
    "version" : "version"
  }, {
    "created" : "created",
    "name" : "name",
    "type" : "type",
    "version" : "version"
  } ],
  "type" : "type",
  "lastUsed" : "lastUsed",
  "customRuntimeImage" : {
    "image" : "image",
    "credential" : "credential",
    "engine" : "engine",
    "created" : "created",
    "modified" : "modified"
  },
  "name" : "name",
  "files" : [ {
    "path" : "path",
    "signature" : "signature",
    "created" : "created",
    "modified" : "modified",
    "sizeBytes" : 0
  }, {
    "path" : "path",
    "signature" : "signature",
    "created" : "created",
    "modified" : "modified",
    "sizeBytes" : 0
  } ],
  "modified" : "modified",
  "retentionPolicy" : "retentionPolicy",
  "status" : "status"
}

Responses

200

OK common.Resource

400

Bad Request util.Response

404

Not Found util.Response

500

Internal Server Error util.Response

Up
get /resources/{name}/{path}
Download a resource file (getResourceFile)
Download a file in the resource at the path specified

Path parameters

name (required)
Path Parameter — Resource name
path (required)
Path Parameter — File resource path

Return type

File

Example data

Content-Type:

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

200

OK File

400

Bad Request util.Response

500

Internal Server Error util.Response

Up
get /resources
List all resources (listResources)
List all resources with their configuration and content

Query parameters

includeFiles (optional)
Query Parameter — Whether to include the files of each resource default: false
filter (optional)
Query Parameter — Filter the list by the syntax 'fieldname[operator]argument'. 'fieldname' is the name of a resource API field. 'operator' is one of ('eq', 'noteq', 'lte', 'lt', 'gte', 'gt', 'in', 'notin', 'like', 'rlike'). For example, 'name[noteq]my-resource'. Multiple filters are ANDed.
limit (optional)
Query Parameter — The maximum number of resources to return, up to 100 default: 20
offset (optional)
Query Parameter — The number of resources to skip before starting list default: 0
orderby (optional)
Query Parameter — The resource API field to order by default: name
orderasc (optional)
Query Parameter — Whether an ordering is ascending default: true

Return type

common.ResourceListResponse

Example data

Content-Type: application/json
{
  "meta" : {
    "offset" : 1,
    "count" : 0,
    "limit" : 6,
    "hasNext" : true
  },
  "resources" : [ {
    "hidden" : true,
    "signature" : "signature",
    "created" : "created",
    "pythonEnv" : {
      "pythonVersion" : "pythonVersion",
      "pyPiMirror" : "pyPiMirror"
    },
    "packages" : [ {
      "created" : "created",
      "name" : "name",
      "type" : "type",
      "version" : "version"
    }, {
      "created" : "created",
      "name" : "name",
      "type" : "type",
      "version" : "version"
    } ],
    "type" : "type",
    "lastUsed" : "lastUsed",
    "customRuntimeImage" : {
      "image" : "image",
      "credential" : "credential",
      "engine" : "engine",
      "created" : "created",
      "modified" : "modified"
    },
    "name" : "name",
    "files" : [ {
      "path" : "path",
      "signature" : "signature",
      "created" : "created",
      "modified" : "modified",
      "sizeBytes" : 0
    }, {
      "path" : "path",
      "signature" : "signature",
      "created" : "created",
      "modified" : "modified",
      "sizeBytes" : 0
    } ],
    "modified" : "modified",
    "retentionPolicy" : "retentionPolicy",
    "status" : "status"
  }, {
    "hidden" : true,
    "signature" : "signature",
    "created" : "created",
    "pythonEnv" : {
      "pythonVersion" : "pythonVersion",
      "pyPiMirror" : "pyPiMirror"
    },
    "packages" : [ {
      "created" : "created",
      "name" : "name",
      "type" : "type",
      "version" : "version"
    }, {
      "created" : "created",
      "name" : "name",
      "type" : "type",
      "version" : "version"
    } ],
    "type" : "type",
    "lastUsed" : "lastUsed",
    "customRuntimeImage" : {
      "image" : "image",
      "credential" : "credential",
      "engine" : "engine",
      "created" : "created",
      "modified" : "modified"
    },
    "name" : "name",
    "files" : [ {
      "path" : "path",
      "signature" : "signature",
      "created" : "created",
      "modified" : "modified",
      "sizeBytes" : 0
    }, {
      "path" : "path",
      "signature" : "signature",
      "created" : "created",
      "modified" : "modified",
      "sizeBytes" : 0
    } ],
    "modified" : "modified",
    "retentionPolicy" : "retentionPolicy",
    "status" : "status"
  } ]
}

Produces

This API call produces the following media types according to the Accept request header; the media type will be conveyed by the Content-Type response header.

Responses

200

OK common.ResourceListResponse

500

Internal Server Error util.Response

Up
put /resources/{name}/{path}
Upload a resource file (putResourceFile)
Upload a new file to the resource in the path specified or replace the existing file resource at the path

Path parameters

name (required)
Path Parameter — Resource name
path (required)
Path Parameter — File resource path

Consumes

This API call consumes the following media types via the Content-Type request header:

Form parameters

file (optional)
Form Parameter — File resource payload

Responses

201

400

Bad Request util.Response

404

Not Found util.Response

500

Internal Server Error util.Response

Models

[ Jump to Methods ]

Table of Contents

  1. common.AirflowRunInfo -
  2. common.AirflowRunSpecRequest -
  3. common.AirflowSpec -
  4. common.AirflowSpecRequest -
  5. common.ArchiveListing -
  6. common.Credential -
  7. common.CredentialCreateRequest -
  8. common.CredentialListResponse -
  9. common.CredentialUpdateRequest -
  10. common.CustomRuntimeImageConfig -
  11. common.DockerBasicCredential -
  12. common.FileResource -
  13. common.Info -
  14. common.Job -
  15. common.JobCreateRequest -
  16. common.JobListResponse -
  17. common.JobResponse -
  18. common.JobUpdateRequest -
  19. common.ListMeta -
  20. common.Package -
  21. common.PythonEnvConfig -
  22. common.Resource -
  23. common.ResourceListResponse -
  24. common.ResourceMountSpec -
  25. common.ResourceRequest -
  26. common.Run -
  27. common.RunJobResponse -
  28. common.RunListResponse -
  29. common.RunLogType -
  30. common.RunOverrides -
  31. common.RunRequest -
  32. common.ScheduleClearRequest -
  33. common.ScheduleMarkSuccessRequest -
  34. common.ScheduleSpec -
  35. common.SparkAppSpec -
  36. common.SparkRunInfo -
  37. common.UserDetail -
  38. util.Response -

common.AirflowRunInfo - Up

config (optional)
dagFile (optional)
dagID (optional)
dagRunID (optional)
executionDate (optional)
startDate (optional)

common.AirflowRunSpecRequest - Up

config (optional)

common.AirflowSpec - Up

config (optional)
dagFile (optional)
dagID (optional)

common.AirflowSpecRequest - Up

config (optional)
dagFile

common.ArchiveListing - Up

adjustedName (optional)
appID (optional)
archiveDirectoryPath (optional)
backupDate (optional)
cdeVersion (optional)
clusterID (optional)
entityType (optional)
envName (optional)
name (optional)
user (optional)

common.Credential - Up

created (optional)
description (optional)
hidden (optional)
metadata (optional)
modified (optional)
name (optional)
type (optional)

common.CredentialCreateRequest - Up

description (optional)
dockerBasic (optional)
hidden (optional)
metadata (optional)
name
type
example: docker-basic

common.CredentialListResponse - Up

credentials (optional)
meta (optional)

common.CredentialUpdateRequest - Up

description (optional)
dockerBasic (optional)
hidden (optional)
metadata (optional)

common.CustomRuntimeImageConfig - Up

created (optional)
credential (optional)
engine
image
modified (optional)

common.DockerBasicCredential - Up

password
server
username

common.FileResource - Up

created (optional)
modified (optional)
path (optional)
signature (optional)
sizeBytes (optional)

common.Info - Up

common.Job - Up

airflow (optional)
created (optional)
hidden (optional)
lastUsed (optional)
modified (optional)
mounts (optional)
name (optional)
retentionPolicy (optional)
runtimeImageResourceName (optional)
schedule (optional)
spark (optional)
type (optional)

common.JobCreateRequest - Up

airflow (optional)
hidden (optional)
mounts (optional)
name
retentionPolicy (optional)
runtimeImageResourceName (optional)
schedule (optional)
spark (optional)
type (optional)

common.JobListResponse - Up

jobs (optional)
meta (optional)

common.JobResponse - Up

airflow (optional)
created (optional)
hidden (optional)
lastUsed (optional)
latestRunInfo (optional)
modified (optional)
mounts (optional)
name (optional)
retentionPolicy (optional)
runtimeImageResourceName (optional)
schedule (optional)
spark (optional)
type (optional)

common.JobUpdateRequest - Up

airflow (optional)
hidden (optional)
mounts (optional)
runtimeImageResourceName (optional)
schedule (optional)
spark (optional)

common.ListMeta - Up

count (optional)
hasNext (optional)
limit (optional)
offset (optional)

common.Package - Up

created (optional)
name (optional)
type (optional)
version (optional)

common.PythonEnvConfig - Up

pyPiMirror (optional)
pythonVersion (optional)

common.Resource - Up

created (optional)
customRuntimeImage (optional)
files (optional)
hidden (optional)
lastUsed (optional)
modified (optional)
name (optional)
packages (optional)
pythonEnv (optional)
retentionPolicy (optional)
signature (optional)
status (optional)
type (optional)

common.ResourceListResponse - Up

meta (optional)
resources (optional)

common.ResourceMountSpec - Up

dirPrefix (optional)
resourceName

common.ResourceRequest - Up

customRuntimeImage (optional)
hidden (optional)
name
pythonEnv (optional)
retentionPolicy (optional)
type (optional)

common.Run - Up

airflow (optional)
ended (optional)
hidden (optional)
id
job
mounts (optional)
runtimeImageResourceName (optional)
spark (optional)
started (optional)
status
type
user (optional)

common.RunJobResponse - Up

id

common.RunListResponse - Up

meta (optional)
runs (optional)

common.RunLogType - Up

entityDefault (optional)
entityLabel (optional)
streamLabel (optional)
type (optional)

common.RunOverrides - Up

airflow (optional)
spark (optional)

common.RunRequest - Up

hidden (optional)
overrides (optional)
user (optional)
variables (optional)

common.ScheduleClearRequest - Up

end (optional)
start (optional)

common.ScheduleMarkSuccessRequest - Up

executionDate (optional)

common.ScheduleSpec - Up

catchup (optional)
cronExpression (optional)
dependsOnPast (optional)
enabled (optional)
end (optional)
nextExecution (optional)
String not stored in DB
paused (optional)
Boolean not stored in DB
pausedUponCreation (optional)
start (optional)
user (optional)

common.SparkAppSpec - Up

args (optional)
className (optional)
conf (optional)
driverCores (optional)
driverMemory (optional)
executorCores (optional)
executorMemory (optional)
file (optional)
files (optional)
jars (optional)
logLevel (optional)
name (optional)
String Livy params
numExecutors (optional)
proxyUser (optional)
pyFiles (optional)
pythonEnvResourceName (optional)

common.SparkRunInfo - Up

batchID (optional)
sparkAppID (optional)
sparkAppURL (optional)
spec (optional)

common.UserDetail - Up

displayName (optional)
username (optional)

util.Response - Up

message (optional)
status (optional)