Verifying replication

To verify the replication, you get a checkpoint event id on the HDP cluster and the last event id. Based on your comparison of the ids, you complete the verification, repeat the verification, or re-replicate the data.

You have run the replication policies, and think you have replicated all databases.
  1. On the HDP cluster backend RDBMS for Hive metastore (HMS), run a query to get a checkpoint event id.
    select NEXT_EVENT_ID - 1 as last_event_id from NOTIFICATION_SEQUENCE;
    Output looks something like this:
    You use the HDP last event id, 582 in this example, as a checkpoint id later.
  2. On the CDP cluster, run the following query to get the last event id for each database replication.
    select name as database_name, param_value as last_event_id 
                                from sys.dbs join sys.database_params on dbs.db_id = database_params.db_id 
                                where param_key='repl.last.id'
    For example, as root you invoke Hive with the Hive 3-supported option -e, and enclose the query exactly as shown above in quotation marks.

    Output looks something like this:

  3. Check the last event ID that appears in the output.
    • If the last event ID equals the HDP checkpoint id, the replication process succeeded for the database. Go to the next step in this procedure.

    • If the last event id is greater than the checkpoint id, verification might be incomplete. Proceed to handling a failed verification.

    • If the last event id is less than the checkpoint id, the replication is still in progress. Continue the replication for that database until the checkpoint id is reached, and reverify.

  4. On the HDP cluster backend RDBMS for Hive metastore (HMS), check that the checkpoint id has not changed since the last time you recorded it. For example
    select NEXT_EVENT_ID - 1 as last_event_id from NOTIFICATION_SEQUENCE;
    Output looks something like this:
  5. Based on your comparison of the last_event_id and the checkpoint ID, proceed in one of the following ways:
    • If the HDP last_event_id does not match the checkpoint id you recorded, go to the next topic, "Handling a failed verification";
    • If the HDP last_event_id does match the checkpoint id, proceed to the next step.
  6. Repeat these steps to verify the replication of each database in the workload, and then validate external table replication.