Cloudera Docs

Using the CDSW to CML Migration tool

A step-by-step procedure covers how to migrate Cloudera Data Science Workbench (CDSW) 1.10.0 and later on premises to Cloudera Machine Learning (CML). You use the UI-driven migration tool to migrate your workload automatically from your deployed CDSW, which you installed on the same cluster as CML.

In this task, you automatically migrate CDSW 1.10.0 or later cluster to CML in Cloudera Private Cloud 1.5.1 or later. You can expect some downtime, which is proportional to the volume the workloads you have to migrate.

Meet prerequisites as described in the previous topic before starting migration.
  1. Log into CDP Private Cloud 1.5.1, and navigate to Cloudera Machine Learning > Workspaces.
    The system detects the presence of your legacy CDSW installation and provides a button to migrate CDSW to CML.
  2. Click Migrate CDSW to CML.
    The Migration tool provides an option to check CML for CDSW migration readiness or just continue with the migration.
  3. Click the option for how you would like to proceed.
    • Check CML for CDSW migration readiness - The readiness or preflight check creates a new workspace and runs readiness checks on the workspace prior to performing the migration.
    • Migrate CDSW to CML - The Migration tool provides a CML workspace provision window with additional options.
  4. If you choose the Check CML for CDSW migration readiness option, the Migration tool displays the validation page.
    1. If you would like to provide a Kubeconfig for the migration check, click File Upload, then Choose File, and select the Kubeconfig file.
      The Kubeconfig file can be found at /etc/kubernetes/admin.conf on the CDSW cluster.
      If you cannot access /etc/kubernetes/admin.conf from the UI as instructed in the previous step, download the file from your CDSW cluster to your local machine, and then try to select the Kubeconfig file from the UI again.
    2. In the Migration timeout section, accept the default 24 hours timeout, or if your CDSW workload is hundreds of gigabytes, increase the migration time up to 336 hours (14 days).
      Increasing the migration timeout value does not cause a delay in the migration of a small workload.
    3. In the Workspace Name field, type an arbitrary name.
    4. In the Select Environment field, select your CDP environment.
      After the readiness check has completed, the Migration tool displays a status. You can see the readiness check summary in the Workspace Details page.
      • If the readiness check fails, you can obtain additional information about the failure on the Workspace Details page.

      • After you've addressed the issue resulting in the failed readiness check, you can retry the readiness check by choosing Retry Migration Readiness check from the Actions menu.

    5. After the readiness check, incremental migration can be performed to continue the migration operation Alternatively, you can choose to create a new workspace by clicking the Migrate CDSW to CML button.
  5. When you proceed with the CDSW to CML migration, the Migration tool displays the Migrations Settings window.
  6. If you would like to provide a Kubeconfig for the migration, click File Upload, then Choose File, and select the Kubeconfig file.
    The Kubeconfig file can be found at /etc/kubernetes/admin.conf on the CDSW cluster.
    If you cannot access /etc/kubernetes/admin.conf from the UI as instructed in the previous step, download the file from your CDSW cluster to your local machine, and then try to select the Kubeconfig file from the UI again.
  7. In the Migration timeout section, accept the default 24 hours timeout, or if your CDSW workload is hundreds of gigabytes, increase the migration time up to 48 hours.
    Increasing the migration timeout value does not cause a delay in the migration of a small workload.
  8. In the Workspace Name field, type an arbitrary name.
  9. In the Select Environment field, select your CDP environment.
  10. Accept default values for other options, and click Provision Workspace.
    After the CML installation, the migration readiness checks and the migration follow automatically. Status indicators show the progress of the installation and migration. During the migration, you can access the CDSW cluster. The migration process does not stop CDSW pods. The CML workspace is stopped.
  11. To display the progress of the migration including events and logs while the workspace is in migration mode, navigate to the Workspace Details page and click the Migration Progress tab.
    You can also view details of the migration and events and logs by clicking the appropriate tabs.
  12. When the initial migration is complete, the state changes to Validate Migration Started.
  13. Open the workspace by clicking the workspace name and validate the workloads.
    Any changes made to the workspace while you are validating the workspace will be overwritten during the incremental migration.
  14. At this point, you can choose to do multiple incremental migrations or a single, longer migration.
    • To perform incremental migrations, select the Incremental CDSW Migration option from the Actions menu.

      CDSW is not stopped during incremental migrations.

    • To perform a single, longer migration, select the Incremental CDSW Migration option from the Actions menu, click the Perform final migration checkbox, then click OK.

      During the final migration, CDSW will be stopped and will not be restarted. After the final migration, only the CML workspace will be active.

    The Migration tool displays the Incremental CDSW migration dialog box.
  15. If you choose to perform incremental migrations, choose your parameters for the migrations.
    CDSW Kubeconfig
    This is an optional parameter. If there is no change to the CDSW Kubeconfig, then you do not need to select this option. The system will use the kubeconfig that was provided during the initial migration. This option can be helpful if CDSW is restarted during the migration and there are changes to the CDSW kubeconfig.
    Migration timeout
    You can specify the amount of time allowed for the migration before it timeouts. This timeout value is specified in hours and can range from 0 to 48 hours.
  16. If you choose to perform incremental migrations, when the CML validation is complete and all workloads work as expected, select Perform final migration and choose the appropriate option.
    After you perform the final migration, CDSW will be in a stopped state.
    Perform final migration provides three options:
    Stop applications
    If you select the Stop applications option, the applications in the CML will be in the stopped state after the migration. You must manually start each application after the final migration.
    Stop jobs
    If you select the Stop jobs option, the recurring jobs in the CML will be in the paused state after the migration. You must manually start each job after the final migration.
    Stop models
    If you select the Stop models option, the models in the CML will be in the stopped state after the migration. You must manually start each model after the final migration.
  17. Now that the migration is complete, you can use CML.