What's new

Cloudera Migration Assistant 4.0 adds major capabilities across Ozone migration, Airflow DAG migration, the CMA Master-Agent model, migration plans and executions, and the web UI, together with numerous bug fixes and a small set of removals and behavior changes.

For defect lists, removals, and behavior changes, see Fixed issues, Deprecations, and Behavioral changes.

HDFS to Ozone migration

Ozone support is significantly enhanced, including the ability to migrate from HDFS to Ozone through Cloudera Replication Manager integration. Scanning discovers volumes, buckets, keys, and snapshot metadata across Ozone. The UI supports mapping and migrating files, collection support for Ozone locations, and detailed display of Ozone bucket and volume attributes.

Ozone Layout Provisioning (technical preview)

CMA can define and deploy Ozone storage layouts (volumes, buckets, and quotas) from the Storage Layout Editor. This workflow is a technical preview and is not ready for production use.

Airflow DAG migration (technical preview)

This release introduces migration of workloads from Cloudera Data Engineering to Cloudera Workflow Orchestrator by automating movement of Airflow DAGs into version-controlled Git repositories. The CMA Agent scans virtual clusters to find Airflow DAGs and their resource pools, organizes them into collections, and maps them to the target repository. An integrated diff editor resolves duplicate DAG IDs and updates environment-specific variables before assets are migrated securely into the target repository without changing the source environment. This workflow is a technical preview and is not ready for production use.

Master-agent architecture

Deployment and security updates introduce the CMA Master-Agent pattern. Agent-based cluster discovery replaces manual UI-based registration. CMA Agents automatically locate their cluster through Cloudera Manager and register with the CMA Master over gRPC. Security enhancements include configurable user and principal in CSDs for agent authentication, use of the Agent's own Kerberos keytab for scanners, and Vault support for CMA Master credential storage.

Migration plans and executions

Cloudera Migration Assistant 4.0 substantially evolves how migrations are modeled and run. Migration plans and executions are separate: plans are versioned and can be run multiple times. Versioned plan collections group datasets and workloads with full history. Versioning extends to structure mappings, workflow configuration, and Hive SQL transformations. Structure mappings are created automatically when items are added to a plan, including Hive, Spark, and Hive SQL. The release also introduces underlying support for incremental migration.

Ranger policy discovery (technical preview)

Scan Ranger policies (access, masking, row-filter) from source clusters. Browse policies in the Discovery browser grouped by Ranger service. Policy indicators on HDFS and Hive resources show authorization coverage. Organize policies into collections for migration planning.

Database support

PostgreSQL support for production deployments (CMA Master, Auth Server, and CMA Agent), with an automated setup script for database provisioning.

UI improvements

Workload visibility is improved in the split view used for mapping. You can create folders on the target side directly in the UI. The execution view adds a plan filter; the Scan UI is revamped; the Cluster Services tab introduces a table view. The plan view displays source and target cluster services and versions, scanning settings are easier to use, and manual cluster creation is hidden in production mode.