Cloudera Docs

Migrating with the inbuilt CDSW to Cloudera AI migration tool

Learn about how to migrate Cloudera Data Science Workbench (CDSW) 1.10.0 and higher versions on premises to Cloudera AI. The migration process is streamlined using a UI-driven migration tool, which automatically transfers your workload from the deployed CDSW instance installed on the same cluster as Cloudera AI.

You can automatically migrate CDSW 1.10.0 or a later cluster to Cloudera AI on premises 1.5.4 or higher versions. You can expect some downtime, which is proportional to the volume of the workloads you have to migrate.

Perform the migration readiness checks before starting migration.
  1. Log into Cloudera on premises, and navigate to Cloudera AI > Workbenches.
    The system detects the presence of your legacy CDSW installation and provides the Migrate CDSW to Cloudera AI button.
  2. Click the Migrate CDSW to Cloudera AI button.
    The Migration tool provides an option to Validate Cloudera AI for CDSW migration readiness or just continue with the migration.
  3. Select an option to proceed.
    • Validate Cloudera AI for CDSW migration readiness - The readiness or preflight validation creates a new workbench and runs readiness validation on the workbench before performing the migration.
    • Migrate CDSW to Cloudera AI - The Migration tool provides a Cloudera AI Workbench provision window with additional options.
  4. If you choose the Validate Cloudera AI for CDSW migration readiness option, the Migration tool displays the validation page.
    1. To provide a Kubeconfig file for the migration check, click the File Upload button, then Choose File, and select the Kubeconfig file.
      The Kubeconfig file can be found at the /etc/kubernetes/admin.conf file on the CDSW cluster.
      If you cannot access the /etc/kubernetes/admin.conf file from the UI as instructed, download the file from your CDSW cluster to your local machine, and then try to select the Kubeconfig file from the UI again.
    2. In the Migration timeout section, accept the default 24 hours timeout, or if your CDSW workload is the size of hundreds of gigabytes, increase the migration time up to 336 hours, that is 14 days.
      Increasing the migration timeout value does not cause a delay in the migration of a small workload.
    3. In the Workbench Name field, type an arbitrary name.
    4. In the Select Environment field, select your Cloudera environment.
      After the readiness validation has completed, the Migration tool displays a status. You can see the readiness validation summary in the Workbench Details page.
      • If the readiness validation fails, you can obtain additional information about the failure on the Workbench Details page.
      • After you have addressed the issue resulting in the failed readiness validation, you can retry the readiness validation by choosing the Retry Migration Readiness Check option from the Actions drop-down list.
    5. After the readiness validation, incremental migration can be performed to continue the migration operation.

      This performs workbench creation and the migration readiness check. If the migration readiness is successful, the migration process will continue with the incremental migration automatically.

  5. Check the Migration Settings window.

    When you proceed with the CDSW to Cloudera AI migration, the Migration tool displays the Migrations Settings window.

    1. To provide a Kubeconfig file for the migration, click the File Upload button, then Choose File, and select the Kubeconfig file.
      The Kubeconfig file can be found at the /etc/kubernetes/admin.conf file on the CDSW cluster.
      If you cannot access the /etc/kubernetes/admin.conf file from the UI as instructed, download the file from your CDSW cluster to your local machine, and then try to select the Kubeconfig file from the UI again.
    2. In the Migration timeout section, accept the default 24 hours timeout, or if your CDSW workload is the size of hundreds of gigabytes, increase the migration time up to 336 hours, that is 14 days.
      Increasing the migration timeout value does not cause a delay in the migration of a small workload.
    3. In the Workbench Name field, type an arbitrary name.
    4. In the Select Environment field, select your Cloudera environment.
    5. Accept default values for other options, and click the Provision Workbench button.
      After the Cloudera AI installation, the migration readiness checks and the migration follow automatically. Status indicators show the progress of the installation and migration. During the migration, you can access the CDSW cluster. The migration process does not stop CDSW pods. The Cloudera AI Workbench is stopped.
    6. Check the status of the migration.

      When the initial migration is complete, the state changes to Validate Migration Started.

    7. Open the workbench by clicking the workbench name and validate the workloads.
      Any changes made to the workbench while you are validating the workbench will be overwritten during the incremental migration and the final migration.
  6. At this point, you can choose to perform multiple incremental migrations or a single, final migration.
    • To perform incremental migrations, select the Incremental CDSW Migration option from the Actions menu.

      CDSW is not stopped during incremental migrations.

    • To perform a single, final migration, select the Incremental CDSW Migration option from the Actions menu, select the Perform final migration checkbox, then click the OK button.

      During the final migration, CDSW will be stopped and will not be restarted. After the final migration, only the Cloudera AI Workbench will be active.

    The Migration tool displays the Incremental CDSW migration dialog box.
  7. If you choose to perform incremental migrations, choose your parameters for the migrations.
    CDSW Kubeconfig
    This is an optional parameter. If no change to the CDSW Kubeconfig occurs, then you do not need to select this option. The system will use the Kubeconfig that was provided during the initial migration. This option can be helpful if CDSW is restarted during the migration and changes to the CDSW Kubeconfig occur.
    Migration timeout
    You can specify the amount of time allowed for the migration before it reaches a timeout. This timeout value is specified in hours and can range from 0 to 48 hours.
  8. If you choose to perform incremental migrations, when the Cloudera AI validation is complete and all workloads work as expected. Select the Perform final migration checkbox and choose the appropriate option.
    After you perform the final migration, CDSW will be in a stopped state.

    You can choose any of the following options for the Perform final migration checkbox:

    Stop applications
    If you select the Stop applications option, the applications in the Cloudera AI will be in the stopped state after the migration. You must start each application manually after the final migration.
    Stop jobs
    If you select the Stop jobs option, the recurring jobs in the Cloudera AI will be in the paused state after the migration. You must restart each job manually after the final migration.
    Stop models
    If you select the Stop models option, the models in the Cloudera AI will be in the stopped state after the migration. You must start each model manually after the final migration.
  9. To display the progress of the migration including events and logs while the workbench is in migration mode, navigate to the Workbench Details page and click the Migration Progress tab.
    You can also view the details of the migration, events and logs by clicking the appropriate tabs.
Now that the migration is complete, you can use Cloudera AI