HashTable/SyncTable tool configuration
You can configure the HashTable/SyncTable tool for your specific needs.
Using the batchsize option
You can define the amount of cell data for a given region that is hashed together in a single
hash value using the
batchsize option, which sets the batchsize property.
Sizing this property has a direct impact on the synchronization efficiency. If the batch size is
increased, larger chunks are hashed.
If only a few differences are expected between the two tables, using a bit larger batch size can be beneficial, as less scans are executed by mapper tasks of SyncTable.
However, if relatively frequent differences are expected between the tables, using a large batch size can cause frequent mismatches of hash values, as the probability of finding at least one mismatch in a batch is increased.
$ hbase org.apache.hadoop.hbase.mapreduce.HashTable --batchsize=32000 --numhashfiles=50 --starttime=1265875194289 --endtime=1265878794289 --families=cf2,cf3 TestTableA /hashes/testTable
Creating a read-only report
You can use the
dryrun option in the second, SyncTable, step to create a read
only report. It produces only COUNTERS indicating the differences between the two tables, but
does not perform any actual changes. It can be used as an alternative of the VerifyReplication
$ hbase org.apache.hadoop.hbase.mapreduce.SyncTable --dryrun=true --sourcezkcluster=zk1.example.com,zk2.example.com,zk3.example.com:2181:/hbase hdfs://nn:8020/hashes/testTable testTableA testTableB
Handling missing cells
By default, SyncTable changes the target table to make it an exact copy of the source table, at least for the specified startrow-stoprow or starttime-endtime.
false modifies the default behaviour to
not delete target cells that are missing on the source table. Similarly, setting
false modifies the default behaviour to not add
missing cells to target table. If you set both
false, the result will be the same as setting
In the case of two-way replication or other scenarios where both source and target clusters
can have data ingested, Cloudera recommends to set
false. Otherwise any additional cells inserted on the SyncTable target cluster
and not yet replicated to the source cluster would be deleted, and potentially lost