Cloudera Docs

Using the CDSW to Cloudera AI Migration tool

A step-by-step procedure covers how to migrate Cloudera Data Science Workbench (CDSW) 1.10.0 and higher version on premises to Cloudera AI. You use the UI-driven migration tool to migrate your workload automatically from your deployed CDSW, which you installed on the same cluster as Cloudera AI.

In this task, you automatically migrate CDSW 1.10.0 or later cluster to Cloudera AI in on premises1.5.3 or later. You can expect some downtime, which is proportional to the volume the workloads you have to migrate.

Meet prerequisites as described in the previous topic before starting migration.
  1. Log into Cloudera on premises, and navigate to Cloudera AI > Workbenches.
    The system detects the presence of your legacy CDSW installation and provides a button to migrate CDSW to Cloudera AI.
  2. Click Migrate CDSW to Cloudera AI.
    The Migration tool provides an option to check Cloudera AI for CDSW migration readiness or just continue with the migration.
  3. Click the option for how you would like to proceed.
    • Check Cloudera AI for CDSW migration readiness - The readiness or preflight check creates a new workbench and runs readiness checks on the workbench prior to performing the migration.
    • Migrate CDSW to Cloudera AI - The Migration tool provides a Cloudera AI Workbench provision window with additional options.
  4. If you choose the Check Cloudera AI for CDSW migration readiness option, the Migration tool displays the validation page.
    1. If you would like to provide a Kubeconfig for the migration check, click File Upload, then Choose File, and select the Kubeconfig file.
      The Kubeconfig file can be found at /etc/kubernetes/admin.conf on the CDSW cluster.
      If you cannot access /etc/kubernetes/admin.conf from the UI as instructed in the previous step, download the file from your CDSW cluster to your local machine, and then try to select the Kubeconfig file from the UI again.
    2. In the Migration timeout section, accept the default 24 hours timeout, or if your CDSW workload is hundreds of gigabytes, increase the migration time up to 336 hours (14 days).
      Increasing the migration timeout value does not cause a delay in the migration of a small workload.
    3. In the Workbench Name field, type an arbitrary name.
    4. In the Select Environment field, select your Cloudera environment.
      After the readiness check has completed, the Migration tool displays a status. You can see the readiness check summary in the Workbench Details page.
      • If the readiness check fails, you can obtain additional information about the failure on the Workbench Details page.
      • After you've addressed the issue resulting in the failed readiness check, you can retry the readiness check by choosing Retry Migration Readiness check from the Actions menu.
    5. After the readiness check, incremental migration can be performed to continue the migration operation Alternatively, you can choose to create a new workbench by clicking the Migrate CDSW to Cloudera AI button.
  5. When you proceed with the CDSW to Cloudera AI migration, the Migration tool displays the Migrations Settings window.
  6. If you would like to provide a Kubeconfig for the migration, click File Upload, then Choose File, and select the Kubeconfig file.
    The Kubeconfig file can be found at /etc/kubernetes/admin.conf on the CDSW cluster.
    If you cannot access /etc/kubernetes/admin.conf from the UI as instructed in the previous step, download the file from your CDSW cluster to your local machine, and then try to select the Kubeconfig file from the UI again.
  7. In the Migration timeout section, accept the default 24 hours timeout, or if your CDSW workload is hundreds of gigabytes, increase the migration time up to 48 hours.
    Increasing the migration timeout value does not cause a delay in the migration of a small workload.
  8. In the Workbench Name field, type an arbitrary name.
  9. In the Select Environment field, select your Cloudera environment.
  10. Accept default values for other options, and click Provision Workbench.
    After the Cloudera AI installation, the migration readiness checks and the migration follow automatically. Status indicators show the progress of the installation and migration. During the migration, you can access the CDSW cluster. The migration process does not stop CDSW pods. The Cloudera AI Workbench is stopped.
  11. To display the progress of the migration including events and logs while the workbench is in migration mode, navigate to the Workbench Details page and click the Migration Progress tab.
    You can also view details of the migration and events and logs by clicking the appropriate tabs.
  12. When the initial migration is complete, the state changes to Validate Migration Started.
  13. Open the workbench by clicking the workbench name and validate the workloads.
    Any changes made to the workbench while you are validating the workbench will be overwritten during the incremental migration.
  14. At this point, you can choose to do multiple incremental migrations or a single, longer migration.
    • To perform incremental migrations, select the Incremental CDSW Migration option from the Actions menu.

      CDSW is not stopped during incremental migrations.

    • To perform a single, longer migration, select the Incremental CDSW Migration option from the Actions menu, click the Perform final migration checkbox, then click OK.

      During the final migration, CDSW will be stopped and will not be restarted. After the final migration, only the Cloudera AI Workbench will be active.

    The Migration tool displays the Incremental CDSW migration dialog box.
  15. If you choose to perform incremental migrations, choose your parameters for the migrations.
    CDSW Kubeconfig
    This is an optional parameter. If there is no change to the CDSW Kubeconfig, then you do not need to select this option. The system will use the kubeconfig that was provided during the initial migration. This option can be helpful if CDSW is restarted during the migration and there are changes to the CDSW kubeconfig.
    Migration timeout
    You can specify the amount of time allowed for the migration before it timeouts. This timeout value is specified in hours and can range from 0 to 48 hours.
  16. If you choose to perform incremental migrations, when the Cloudera AI validation is complete and all workloads work as expected, select Perform final migration and choose the appropriate option.
    After you perform the final migration, CDSW will be in a stopped state.
    Perform final migration provides three options:
    Stop applications
    If you select the Stop applications option, the applications in the Cloudera AI will be in the stopped state after the migration. You must manually start each application after the final migration.
    Stop jobs
    If you select the Stop jobs option, the recurring jobs in the Cloudera AI will be in the paused state after the migration. You must manually start each job after the final migration.
    Stop models
    If you select the Stop models option, the models in the Cloudera AI will be in the stopped state after the migration. You must manually start each model after the final migration.
  17. Now that the migration is complete, you can use Cloudera AI.