Post-upgrade Tasks
Atlas
The settings for the Atlas migration need to be removed once the upgrade has been completed.
From the Ambari Web UI, select Atlas > Configs > Advanced > Custom application-properties, then click the red Remove button to remove the atlas.migration.data.filename property. Restart Atlas when prompted by Ambari.
Ambari Metrics, SmartSense, LogSearch
Take the Ambari Metrics, SmartSense, and Log Search Services out of Maintenance Mode by choosing Actions > Turn Off Maintenance Mode from each Service page.
Ambari Infra-Migrate & Restore
Follow the steps below to restore the data previously backed up:
SSH to the host where the
migrationConfigGenerator.py
was run prior to the HDP Upgrade. This will be from one of your Ambari Infra Solr instances. Please ensure you are in the current working directory containing theambari_solr_migration.ini
file.Export the variable used to hold the path to the ini file.
export CONFIG_INI_LOCATION=ambari_solr_migration.ini
Migrate the data to ensure it’s in the right format to import into Solr 7. Please note that this script can take a long time to run depending on the volume of data backed up. It is recommended to run this script using the nohup command.
nohup /usr/lib/ambari-infra-solr-client/ambariSolrMigration.sh --ini-file $CONFIG_INI_LOCATION --mode migrate-restore > migrate_restore_output.txt 2>&1 &
To observe the progress of the migrate and restore process, simply tail the migrate_restore_output.txt file.
tail -f migrate_restore_output.txt
Re-index the migrated data into your current collections so the backed up data is visible in all of the tools using the Infra Solr instances. Please note that this script can take a long time to run depending on the volume of data backed up. It is recommended to run this script using the nohup command.
nohup /usr/lib/ambari-infra-solr-client/ambariSolrMigration.sh --ini-file $CONFIG_INI_LOCATION --mode transport > transport_output.txt 2>&1 &
To observe the progress of the transport process, simply tail the transport_output.txt file:
tail -f transport_output.txt
Migrate Ambari Metrics Data
Use the following steps to migrate data from the previous AMS schema to the new AMS schema.
Ensure the Ambari Metrics System is started. If it is not, in the Ambari Web UI, click Ambari Metrics, then select Actions > Start.
SSH into a host that is running a Metrics Collector. You can locate this host by going to the Ambari Web UI and clicking Hosts. Click on the Filter icon and type in "Metrics Collector: All" to find each host that has a Metrics Collector installed on it.
SU to the Ambari Metrics user. This is 'ams' by default, but if you don’t know which user is configured in your cluster go to the Ambari Web UI and click Cluster Admin > Service Accounts, and then look for "Ambari Metrics User".
# su ams
Run the command to migrate data from the old Ambari Metrics schema to the new.
$ /usr/sbin/ambari-metrics-collector --config /etc/ambari-metrics-collector/conf/ upgrade_start /etc/ambari-metrics-collector/conf/metrics_whitelist
Note The default time period for data migration is one month (2592000000 milli seconds), which can be overwritten if required. For overwriting the data migration to more than one month, you must provide the timeframe upto which data need to be migrated along with the command.
/usr/sbin/ambari-metrics-collector --config /etc/ambari-metrics-collector/conf/ upgrade_start /etc/ambari-metrics-collector/conf/metrics_whitelist "31556952000"
"31556952000" is the amount of days in milliseconds to be preserved and 31556952000 stands for 1 year.
Once the upgrade process has started, logs are available in the <ams-log-dir>/ambari-metrics-migration.log file.
Update Ranger Passwords
Ranger password validation has been updated in HDP 3.0, and to conform to these new password policies, the following Ranger passwords need to be updated to ensure that they have at least 8 characters with minimum one alphabet and one numeric. These passwords cannot contain the following special characters: " ' \ `
Ranger Admin
The following new passwords need to be populated with valid passwords that also conform to the password policy:
Ranger Usersync User’s Password
Ranger Tagsync User’s Password
Ranger KMS Keyadmin User’s Password