Cloudera Docs

Understanding how DDL commands affect Hive tables during replication

Before you create Hive replication policies, you must understand how DDL commands affect the Hive tables during replication.

The following scenarios explain how the tables are affected when you use the drop table and truncate table DDL commands on Hive external tables in a Hive external table replication policy:
  • You drop a table in a replication policy after the policy has run at least once. The table remains on the destination cluster and does not get dropped during subsequent replication runs.
  • You rename a table on the source cluster. On the target cluster, a table is created with the new name and the old table is also retained.
  • You drop a table on the destination cluster and the table is still included in the replication job. The table is re-created on the destination during the next replication job.
  • You drop a table partition or index on the source cluster. The next replication job drops it on the destination cluster.
  • You truncate a table, and the Advanced > Delete Policy field is set to Delete to Trash or Delete Permanently in the Hive external table replication policy wizard. The corresponding data files are deleted on the destination during the next replication job.

The following table shows the resulting actions in the target cluster when you run DDL commands in the source cluster on the Hive external tables used in an Hive external table replication policy:

DDL command on Hive external tables in source cluster Resulting actions on the table/partition and the folder it resides on the source cluster Resulting actions on the table/partition and the folder it resides on the target cluster after replication
create table
  • Creates table / partition
  • Creates folder
  • Creates table
  • Creates folder
drop table
  • Drops table
  • Retains the folder
  • Retains table
  • Retains folder
drop table with 'external.table.purge'='true'
  • Drops table
  • Deletes folder
  • Retains table
  • Retains folder
rename table
  • Renames table
  • Retains folder name
  • Creates a table with the new name and the original table is retained.
  • The folder is retained and houses newly created and original tables.
add partition
  • Creates partition
  • Creates folder
  • Creates partition
  • Creates folder
drop partition
  • Drops partition
  • Retains folder
  • Drops partition
  • Retains folder
drop partition with 'external.table.purge'='true'
  • Drops partition
  • Deletes folder
  • Drops partition
  • Retains folder
rename partition
  • Renames partition
  • Retains folder
  • Renames partition
  • Retains folder