Migrating from the Cloudera Manager External PostgreSQL Database Server to a MySQL/Oracle Database Server

Cloudera Manager provides an embedded PostgreSQL database server for demonstration and proof of concept deployments when creating a cluster. To remind users that this embedded database is not suitable for production, Cloudera Manager displays the banner text: "You are running Cloudera Manager in non-production mode, which uses an embedded PostgreSQL database. Switch to using a supported external database before moving into production."

If you have already used the embedded database, and you are unable to redeploy a fresh cluster, then you must migrate to an external PostgreSQL database. For details about how to migrate from an embedded PostgreSQL database to an external PostgreSQL database, refer to Migrating from the Cloudera Manager Embedded PostgreSQL Database Server to an External PostgreSQL Database.

Prerequisites

Before migrating from the Cloudera Manager external PostgreSQL database to an external MySQL/Oracle database, ensure that your setup meets the following conditions:
  • Configuration uses Cloudera Manager 5.15.0 or later on supported platforms.
  • You must have a valid Cloudera Manager Enterprise license.
  • If Cloudera Manager is secured, then you must import Kerberos account manager credentials and regenerate them. For details, refer to Step 3: Add the Credentials for the Principal to the Cluster.
  • You must have a destination host installed with the supported database of choice (MySQL or Oracle). For details about installing and configuring MySQL for Cloudera, see Install and Configure MySQL for Cloudera Software. For details about installing and configuring Oracle for Cloudera, see Install and Configure Oracle Database for Cloudera Software.
  • You have made configured target database hosts available.
  • You have planned for cluster downtime during the migration process.
  • You have a plan to follow service specific database migration instructions for services other than Cloudera Manager. Refer to the appropriate service migration documentation for your cluster setup (for example, Migrate the Hue Database).
  • No one has manually created any databases in the external database server for roles that will be migrated.
  • All health issues with your cluster are resolved.

For large clusters, Cloudera recommends running your database server on a dedicated host. Engage Cloudera Professional Services or a certified database administrator to correctly tune your external database server.

Migrate from the Cloudera Manager External PostgreSQL Database Server to a MySQL/Oracle Database Server

To migrate from the Cloudera Manager external PostgreSQL database server to a MySQL or Oracle database server:
  1. Migrate from the embedded PostgreSQL database server to an external PostgreSQL database server as described in Migrating from the Cloudera Manager Embedded PostgreSQL Database Server to an External PostgreSQL Database. Migrating directly from the Cloudera Manager embedded PostgreSQL to a MySQL/Oracle database is not supported. You must first migrate from the Cloudera Manager embedded PostgreSQL database server to the external PostgreSQL database server. After performing this migration, you can use this procedure to migrate from the external PostgreSQL database server to MySQL/Oracle database servers.
  2. Export your Cloudera Manager Configuration. First, get the latest supported API version: 
    curl -u <admin_username>:<admin_password> "http://<cm_server_host>:7180/api/version"
    curl -u <admin_username>:<admin_password> "http://<cm_server_host>:7180/api/<api_version> /cm/deployment" > <path_to_file>/cm-deployment.json
    Following is an example of the API version command: 
    curl -u admin:admin "http://10.17.103.191:7180/api/v19/cm/deployment" > /root/cm-deployment.json
  3. Preserve Cloudera Manager's GUID by running the following command to create a /etc/cloudera-scm-server/uuid file. On a host that has an agent, run: 
    sudo -u postgres psql -qtAX scm -c "select GUID from CM_VERSION" > uuid

    Then move the UUID file to Cloudera Manager server's /etc/cloudera-scm-server directory.

  4. Stop the cluster and the Cloudera Management Services. For details on how to stop services, see Starting, Stopping, and Restarting Services.
  5. Stop the Cloudera Manager Server: 
    sudo service cloudera-scm-server stop
  6. Prepare the target database for Cloudera Manager. For details, refer to Install and Configure MySQL for Cloudera Software or Install and Configure Oracle Database for Cloudera Software.
  7. The process directory (/var/run/cloudera-scm-agent/process/) must be cleaned out for all of the hosts that have agents running on them. The agent completes this cleanup with a server reboot. However, if a server reboot is not a viable option, use one of the following options to accomplish the same task.
    1. Stop the agent and supervisor: 
      service cloudera-scm-agent hard_stop
    2. Confirm that the agent and supervisor process are stopped: 
      ps -ef | grep -i cmf-agent; ps -ef | grep -i supervisor
    3. Perform a clean start: 
      service cloudera-scm-agent next_start_clean
      Alternatively, run the following command to view the start options available on your platform: 
      service cloudera-scm-agent clean_start
    4. Ensure that the process is empty: 
      ls -la /var/run/cloudera-scm-agent/process/

    The other remaining option for cleanup:

    1. Stop the agent and supervisor: 
      service cloudera-scm-agent hard_stop
    2. Confirm that the agent and supervisor process are stopped: 
      ps -ef | grep -i cmf-agent; ps -ef | grep -i supervisor
    3. Move the existing /var/run/cloudera-scm-agent/ directory: 
      mv /var/run/cloudera-scm-agent /var/run/cloudera-scm-agent-BU

    The agent will recreate the directory. Delete the backed up copy after confirming that the migration was successful.

  8. Start the Cloudera Manager server: 
    service cloudera-scm-server start
  9. Log in to Cloudera Manager. Exit the installation wizard by clicking the product log in the upper-left corner to stop the wizard and return to the Cloudera Manager home page.
  10. Upgrade the Cloudera Manager Enterprise License by navigating to Administration > Licenses and installing a valid Cloudera Manager license.
  11. Restore the Cloudera Manager configuration: 
    curl -H "Content-Type: application/json" --upload-file <path_to_file>/cm-deployment.json -u <admin_username>:<admin_password> "http://<cm_server_host>:7180/api/<api_version>/cm/deployment?deleteCurrentDeployment=true"
    The following example shows how to restore a Cloudera Manager configuration: 
    curl -H "Content-Type: application/json" --upload-file /root/cm-deployment.json -u admin:admin "http://172.31.113.146:7180/api/v19/cm/deployment?deleteCurrentDeployment=true"
  12. Start the following: Cloudera Management Service, Host Monitor, and Services Monitor. Verify that all the services in the Cloudera Management Service started and are Healthy.
  13. Select the Home > Status tab for the cluster(s) that you previously stopped, and in the Actions dropdown, select Start.