Cloudera Data Engineering resources
A resource in Cloudera Data Engineering (CDE) is a
named collection of files used by a job. Resources can include application
code, configuration files, Python virtual environment specifications
requirements.txt). A resource can also be a custom
Docker container image.
Resources are associated with virtual clusters. A resource can be used by
multiple jobs, and jobs can use multiple resources.The resource types
supported by CDE are
- An arbitrary collection of files that a job can reference. The
application code for the job, including any necessary configuration
files or supporting libraries, can be stored in a
filesresource. Files can be uploaded to and removed from a resource as needed.
- A defined virtual Python environment that a job runs in. The only
file that can be uploaded to a
python-envresource is a
requirements.txtfile. When you associate a
python-envresource with a job, the job runs within a Python virtual environment built according to the
- A Docker container image. When you run a job using a
custom-runtime-imageresource, the executors that are launched use your custom image.