Updating Python virtual environment resource

Python virtual environments cannot be updated. Instead, create a new python-env resource and update the job to reference the new resource.

  • Download and configure the CDE CLI.
  • Create a new requirements.txt file specifying the Python package and version dependencies required by your CDE job.
  1. Create a new python-env resource.
    cde resource create --name new-cde-python-env-resource --type python-env --python-version python3
  2. Upload the new or updated requirements.txt file to the new resource.
    cde resource upload --name new-cde-python-env-resource --local-path ${HOME}/requirements.txt
  3. Update the CDE job to specify the new resource.
    cde job update --name pyspark-example --python-env-resource-name new-cde-python-env-resource