Upgrading HDF to CFM on CDP Private Cloud Base

Cloudera provides an AM2CM tool to enable you to upgrade HDF cluster to CFM cluster on CDP Private Cloud Base easily. You need to download the AM2CM tool, configure settings for the tool, and generate Cloudera Manager template. After you generate the Cloudera Manager template, you need to configure the parcel, deploy Cloudera Manager, add CFM parcel, and activate the parcel through the Cloudera Manager UI.

  1. Ensure that you have the latest Ambari blueprint (JSON file).
  2. Download the AM2CM tool:
    wget https://archive.cloudera.com/am2cm/
  3. Extract the TAR file:
    tar -xvf am2cm-tool-
  4. Check the project files:
    # cd am2cm-
                            # [root@ccycloud-1 am2cm-]# ls
    am2cm-  am2cm.sh  conf  lib  ranger_policy_migration.py
    # [root@ccycloud-1 am2cm-]# ls conf
    ambari_blueprint_hostgroups.josn  blueprint.json  cm-config-mapping-custom.ini  cm-config-mapping.ini  log4j.properties  ranger_policy_migration.py  service-config.ini  service-default-config.ini  user-settings.ini
    # mv conf/blueprint.json conf/blueprint_initial.json
    # [root@ccycloud-1 am2cm-]# ls conf
    ambari_blueprint_hostgroups.josn  blueprint_initial.json  cm-config-mapping-custom.ini  cm-config-mapping.ini  log4j.properties  ranger_policy_migration.py  service-config.ini  service-default-config.ini  user-settings.ini
  5. Set JAVA_HOME if not already done:
    [root@ccycloud-1 am2cm-]# echo $JAVA_HOME
  6. Copy your Ambari blueprint to the right path, that is the conf directory of the AM2CM tool, from the Home directory:
    cp /data/backups/HDFTOCDF_blueprint_with_hosts_20201103144235.json /root/am2cm-
  7. Configure the user-settings.ini file:

    It is really important to specify the cluster.name and cluster.displayname to the same original Ambari cluster name. Also, it is worth to fill all password field.

    # Cluster details
    # Update this with correct CM/CDP version before running the tool
    cluster.originating_source=AM2CM version 2.0
    # This is needed to update Ranger/Atlas policy service names.
    #This flag enables to translate Config Groups to Role config groups.
    cm.rolegroups.enable = false
    # FROM JDBC URL - tool reads only Hostname and port number - rest of the details like Database type, Database name, database user reads from from Blueprint.
    # Do not enclose double quotes for passwords, provide exact password.
    # Hive JDBC settings
    # Oozie JDBC settings
    # Ranger JDBC settings
    #Knox Settings
  8. If your cluster contains Kafka service, please override the kafka-broker-ids.ini files with the one you created.
    For more details, see Extracting Kafka broker IDs.
  9. Run the command to generate the Cloudera Manager template:
    # cd am2cm-
    # chmod +x ./am2cm.sh
    [root@ccycloud-1 am2cm-]# ./am2cm.sh -bp conf/HDFTOCDF_blueprint_with_hosts_20201103144235.json -dt conf/cm_deployment_template.json --source-version=HDF352
    INPUT Ambari Blueprint : conf/HDFTOCDF_blueprint_with_hosts_20201103144235.json
    OUTPUT CM Template     : conf/cm_deployment_template.json
    Starting blueprint to CM Template migration
    Total number of hosts in blueprint: 4
    Your cluster has services (listed below) that are not handled by this migration tool.
    The tool will skip the above identified service related configs.
    ** This migration tool is a technical preview only **
    Do you want to proceed with migration (Y OR N)? (N):
    Processing: LIVY
    Processing: SOLR
    Processing: TEZ
    Processing: HDFS
    Processing: OOZIE
    Processing: SQOOP_CLIENT
    Processing: NIFIREGISTRY
    Processing: ZOOKEEPER
    Processing: HBASE
    Processing: YARN
    Processing: RANGER_KMS
    Processing: KNOX
    Processing: ATLAS
    Processing: HIVE_ON_TEZ
    Processing: RANGER
    Processing: HIVE
    Processing: KAFKA
    Processing: NIFI
    Processing: SPARK_ON_YARN
    CM Template is generated at : /root/am2cm-
    Successfully completed
  10. Check for errors in the tool log for Cloudera Manager migration:
    less am2cm-