Migrating Cloudera Manager Management Services to Another Host
Relocating and migrating data from Cloudera Manager Management Services to another
host.
To move the existing Cloudera Manager Management Services to another host. You must have the minimum requirements to proceed with the migration process.
Cloudera Manager 5 and higher
Database credentials are required for the configured RDBMS
databases.
Password information
Embedded database
installations create passwords on the disk that can be collected in
/etc/cloudera-scm-server/db.mgmt.properties. You must have
these credentials to reapply the database configuration during the role
move.
Directory locations
Default
Management data directory locations are used in all commands, adjust your location
as required for the process.
You can read more about how to determine these values here:
These documents contain information about the data storage types and
requirements discussed in this process.
You should understand them
well so that you can properly size a new host for the Cloudera Management Service
processes and data.
Record the existing service database credentials like:
hostname
type
dbuser
dbpass
configuration values (heap and safety values)
From Cloudera Manager, navigate to > Clusters > Cloudera Management Services >
Configuration> Reports Manager > Database
From Cloudera Manager navigate to Clusters > Cloudera Management Services >
Configuration > Alert Publisher
Record the values for the following:
Mail Server Hostname
Username
Password
Recipients
Confirm if the Java Keystore (If TLS is enabled).
From Cloudera Manager, navigate to Clusters > Cloudera Management Services >
Cloudera Management Service (Service-Wide) > Security > SSL Client Truststore
File Location.
Copy Truststore file to the new HostB node in the same directory location,
owner as
cloudera-scm:cloudera-scm
[HostA]# cd /var/lib/cloudera-scm-agent/agent-cert
[HostA]# tar -czpf /tmp/agent-cert.tar.gz agent-cert
[HostA]# scp /tmp/agent-cert.tar.gz HostB.example.com:/tmp
[HostB]# cd /var/lib/cloudera-scm-agent/agent-cert
[HostB]# tar -xf /tmp/agent-cert.tar.gz
Stop all the Cloudera Manager Management services: Cloudera Manager > Clusters
> Cloudera Management Services > Actions > Stop
Relocate the data directories to the new host with the same permissions and
owner.
cloudera-scm:cloudera-scm
Issue the commands on HostA to tar up preserving ownership and perms, later
securely copy (SCP) the service data to Host B - (you might need to SCP the
files between servers using another method.)
[HostA]# cd /var/lib
[HostA]# tar -czpf /tmp/cloudera-host-monitor-data-dir.tar.gz cloudera-host-monitor
[HostA]# tar -czpf /tmp/cloudera-service-monitor-data-dir.tar.gz cloudera-service-monitor
[HostA]# tar -czpf /tmp/cloudera-scm-eventserver-data-dir.tar.gz cloudera-scm-eventserver
[HostA]# tar -czpf /tmp/cloudera-scm-navigator-data-dir.tar.gz cloudera-scm-navigator
[HostA]# scp /tmp/*-data-dir.tar.gz HostB.example.com:/tmp
Issue the commands on Host B to uncompress the tarball into
/var/lib on the new host.
[HostB]# cd /var/lib
[HostB]# tar -xf /tmp/*-data-dir.tar.gz
Review each service and apply any remaining configuration specifics like Heap
or safety valves noted earlier.
Click Finish to save the various changes.
Start all the Cloudera Manager Management services
Confirm health and functionality of each service.
Wait five minutes or more and the graphs should have data and the cluster
status should be visible. Allow Service monitor time to run canary tests, some
errors require several sequential positive results.
Use Cloudera Manager to refresh the pages and the graph data should
reappear.
Cleanup and remove the old data and log directories from old hosts
(HostA).
Default Locations of Cloudera Management Service
logs and data directories
Index of Cloudera Management Service default locations for logs and data
directories:
Event Server Index Directory
(/var/lib/cloudera-scm-eventserver)
Service Monitor Storage Directory
(/var/lib/cloudera-service-monitor)
Alert Publisher has no on disk data, only log location and alert config
required during Add Role Wizard
Reports Manager Working Directory
(/var/lib/cloudera-scm-headlamp)
Reports Manager data in RDBMS, reconfiguration required during Add Role
Wizard (working directory is temporary location for hourly processing,
no data is stored there)
Service Monitor Log Directory
(/var/log/cloudera-scm-firehose)
Steps to change the passwords for a management service in embedded
Postgres.
# export PGPASSWORD=$(head -1 /var/lib/cloudera-scm-server-db/data/generated_password.txt)
# psql -U cloudera-scm -h localhost -p 7432 scm
scm=# ALTER USER rman with password 'rman';
ALTER ROLE
scm=# \q