Cloudera Docs

Migrating Cloudera Management Services to Another Host

Relocating and migrating data from Cloudera Management Services to another host.

To move the existing Cloudera Management Services to another host.
You must have the minimum requirements to proceed with the migration process.
  • Cloudera Manager 5 and higher
  • Database credentials are required for the configured RDBMS databases.

Password information

Embedded database installations create passwords on the disk that can be collected in /etc/cloudera-scm-server/db.mgmt.properties. You must have these credentials to reapply the database configuration during the role move.

Directory locations

Default Management data directory locations are used in all commands, adjust your location as required for the process.

You can read more about how to determine these values here:

These documents contain information about the data storage types and requirements discussed in this process.

You should understand them well so that you can properly size a new host for the Cloudera Management Service processes and data.

  1. Record the existing service database credentials like:
    • hostname
    • type
    • dbuser
    • dbpass
    • configuration values (heap and safety values)
  2. From Cloudera Manager, navigate to Clusters > Cloudera Management Services > Configuration > Reports Manager > Database.
  3. From Cloudera Manager navigate to Clusters > Cloudera Management Services > Configuration > Alert Publisher.
    Record the values for the following:
    • Mail Server Hostname
    • Username
    • Password
    • Recipients
    Confirm if the Java Keystore (If TLS is enabled).
  4. From Cloudera Manager, navigate to Clusters > Cloudera Management Services > Cloudera Management Service (Service-Wide) > Security > SSL Client Truststore File Location.
  5. Copy Truststore file to the new HostB node in the same directory location, owner as 
    cloudera-scm:cloudera-scm

[HostA]# cd /var/lib/cloudera-scm-agent/agent-cert

[HostA]# tar -czpf /tmp/agent-cert.tar.gz agent-cert 

[HostA]# scp /tmp/agent-cert.tar.gz HostB.example.com:/tmp

[HostB]# cd /var/lib/cloudera-scm-agent/agent-cert

[HostB]# tar -xf /tmp/agent-cert.tar.gz
  6. Stop all the Cloudera Management Services: navigate to Cloudera Manager > Clusters > Cloudera Management Services > Actions > Stop.
  7. Relocate the data directories to the new host with the same permissions and owner. 
    cloudera-scm:cloudera-scm
    Issue the commands on HostA to tar up preserving ownership and perms, later securely copy (SCP) the service data to Host B - (you might need to SCP the files between servers using another method.)
    [HostA]# cd /var/lib

[HostA]# tar -czpf /tmp/cloudera-host-monitor-data-dir.tar.gz cloudera-host-monitor

[HostA]# tar -czpf /tmp/cloudera-service-monitor-data-dir.tar.gz cloudera-service-monitor

[HostA]# tar -czpf /tmp/cloudera-scm-eventserver-data-dir.tar.gz cloudera-scm-eventserver

[HostA]# tar -czpf /tmp/cloudera-scm-navigator-data-dir.tar.gz cloudera-scm-navigator

[HostA]# scp /tmp/*-data-dir.tar.gz HostB.example.com:/tmp
  8. Issue the commands on Host B to uncompress the tarball into /var/lib on the new host. 
    [HostB]# cd /var/lib

[HostB]# tar -xf /tmp/*-data-dir.tar.gz
  9. Remove the roles: navigate to Cloudera Manager > Clusters > Cloudera Management Services > Instances.
    Select all > Actions for Selected > Delete
  10. Click Add Role instances.
  11. Select the specific host under each service to assign the roles to nodes and click Next to continue.
    Mandatory Roles
    Some of the roles that are required for you to proceed further with the migration process.
    • Event Server
    • Alert Publisher
    • Host Monitor
    • Service Monitor
    • Reports Manager
  12. Input and test any of the database credential collected from Step #1.

    For example, for the embedded database on HostA, the configuration for Rman would be as follows:

    Database Host Name: HostA.example.com:7432

Database Type: PostgreSQL

Database Name : rman

Username: rman

Password: rman
  13. Review each service and apply any remaining configuration specifics like Heap or safety valves noted earlier.
  14. Click Finish to save the various changes.
  15. Start all the Cloudera Management Services.
  16. Confirm health and functionality of each service.
    Wait five minutes or more and the graphs should have data and the cluster status should be visible. Allow Service monitor time to run canary tests, some errors require several sequential positive results.
  17. Use Cloudera Manager to refresh the pages and the graph data should reappear.
  18. Cleanup and remove the old data and log directories from old hosts (HostA).
    Default Locations of Cloudera Management Service logs and data directories
    Index of Cloudera Management Service default locations for logs and data directories:
    • Event Server Index Directory (/var/lib/cloudera-scm-eventserver)
    • Host Monitor Storage Directory (/var/lib/cloudera-host-monitor)
    • Service Monitor Storage Directory (/var/lib/cloudera-service-monitor)
    • Alert Publisher has no on disk data, only log location and alert config required during Add Role Wizard
    • Reports Manager Working Directory (/var/lib/cloudera-scm-headlamp)
    • Reports Manager data in RDBMS, reconfiguration required during Add Role Wizard (working directory is temporary location for hourly processing, no data is stored there)
    • Alert Publisher Log Directory (/var/log/cloudera-scm-alertpublisher)
    • Event Server Log Directory (/var/log/cloudera-scm-eventserver)
    • Host Monitor Log Directory (/var/log/cloudera-scm-firehose)
    • Audit Log Directory (/var/log/cloudera-scm-navigator/audit)
    • Reports Manager Log Directory (/var/log/cloudera-scm-headlamp)
    • Service Monitor Log Directory (/var/log/cloudera-scm-firehose)
    Steps to change the passwords for a management service in embedded Postgres.
    # export PGPASSWORD=$(head -1 /var/lib/cloudera-scm-server-db/data/generated_password.txt)

# psql -U cloudera-scm -h localhost -p 7432 scm

scm=# ALTER USER rman with password 'rman';

ALTER ROLE

scm=# \q