Verifying replication
To verify the replication, you get a checkpoint event id on the HDP cluster and the last event id. Based on your comparison of the ids, you complete the verification, repeat the verification, or re-replicate the data.
-
On the HDP cluster backend RDBMS for Hive metastore (HMS), run a query to get a checkpoint event id.
select NEXT_EVENT_ID - 1 as last_event_id from NOTIFICATION_SEQUENCE;
Output looks something like this: You use the HDP last event id, 582 in this example, as a checkpoint id later. -
On the CDP cluster, run the following query to get the last event id for each database replication.
select name as database_name, param_value as last_event_id from sys.dbs join sys.database_params on dbs.db_id = database_params.db_id where param_key='repl.last.id'
For example, as root you invoke Hive with the Hive 3-supported option -e, and enclose the query exactly as shown above in quotation marks. -
Check the last event ID that appears in the output.
-
If the last event ID equals the HDP checkpoint id, the replication process succeeded for the database. Go to the next step in this procedure.
-
If the last event id is greater than the checkpoint id, verification might be incomplete. Proceed to handling a failed verification.
-
If the last event id is less than the checkpoint id, the replication is still in progress. Continue the replication for that database until the checkpoint id is reached, and reverify.
-
-
On the HDP cluster backend RDBMS for Hive metastore (HMS), check that the checkpoint id has not changed since the last time you recorded it. For example
select NEXT_EVENT_ID - 1 as last_event_id from NOTIFICATION_SEQUENCE;
Output looks something like this: -
Based on your comparison of the last_event_id and the checkpoint ID, proceed in one of the following ways:
- If the HDP last_event_id does not match the checkpoint id you recorded, go to the next topic, "Handling a failed verification";
- If the HDP last_event_id does match the checkpoint id, proceed to the next step.
- Repeat these steps to verify the replication of each database in the workload, and then validate external table replication.