Uploading files or other assets to a Cloudera Data Engineering resource using the CLI

A resource in Cloudera Data Engineering (CDE) is a named collection of files or other assets referenced by a job, including application code, configuration files, or any other arbitrary files required by a job. A resource can also be a Python virtual environment, or a custom Docker container image.

Make sure that you have downloaded and configured the CLI client.

Make sure that you have created a resource.

  1. Upload assets to a resource using the cde resource upload command.
    The cde resource upload syntax is as follows:
    cde resource upload [flags]

    You can view the list of flags by running cde resource upload --help, or you can view the CDE CLI reference documentation.

    Example: Upload a file resource
    cde resource upload --name [***RESOURCE_NAME***] --local-path [***LOCAL_PATH***] [--resource-path [***PATH_IN_RESOURCE***]]

    Use repeated local path flags, and/or */?/[a-z] wildcards, to specify multiple files. Use quotes around the local path when including wildcards, for example, --local-path "*.jar". For a single file --resource-path is used for the resource filename. For multiple files --resource-path is used for the resource directory.

    Example: Upload a Python virtual environment resource
    cde resource upload --name cde-python-env-resource --local-path ${HOME}/requirements.txt
    Example: Upload a custom Docker container image resource
    cde resource upload --name cde-container-image-resource --type custom-runtime-image
    Example: Upload a file for a file resource
    cde resource upload --name cde-file-resource --local-path /path/to/local/file
    Example: Upload and extract archive to resource

    Currently supported archive file formats are : .zip and .tar.gz

    cde resource upload-archive --name cde-file-resource --local-path /path/to/local/file
  2. Verify that the file is included in the resource by running cde resource describe --name <resource_name>.