Restoring jobs and Airflow Operators and Libraries from a CDE version 1.20.3 VC backup

In CDE version 1.20.3, the backup and restore function was not implemented yet for Airflow Operators and Libraries. To restore a CDE version 1.20.3 Virtual Cluster (VC) from a backup, first create the VC manually, after that create and activate the Airflow Operators and Libraries with the same requirements.txt file that the original VC used, and restore the jobs, resources, and so on from the archive.

Make sure that you have created a backup on a local storage from a CDE version 1.20.3 VC with active state Airflow Operators and Libraries with the following command:

cde backup create --include-jobs --include-resources
                --include-credential-secrets --include-credentials --local-path
                backup.zip

Optionally, you can validate the backup.

cde backup
                list-archive --local-path backup.zip

Create a VC manually with the same configuration (Spark version, CPU, memory, and so on) as the original VC.
For more information, see Creating virtual clusters.
Create and activate the Airflow Operators and Libraries manually. Use the same requirements.txt file that the backed up VC used.
For more information about obtaining the requirements.txt file, see In-place upgrade with Airflow Operators and Libraries.

Restore the jobs, resources, and so on from the archive.

cde backup restore --local-path backup.zip

Example:

$ cde backup restore --local-path backup.zip
     8.2KB/8.2KB 100% [==============================================] backup.zip
All items restored successfully! Details:
Successfully restored items:
  Job:helloworld ( Created )
  Job:sparkhello ( Created )
  Resource:files ( Created )
  Resource:cde-airflow-pyenv-1711096070 ( Created )