Hortonworks Docs » Data Lifecycle Manager 1.5.1 » DLM Administration

DLM Administration

Also available as:

PDF

loading table of contents...

Introduction
- Purpose and scope
- Audience and assumptions
Replication concepts
UI overview
Preparing to setup replication policy
- Roles required
- Working with clusters
  - Add clusters
Cloud credentials
- Register cloud credentials
Data replication use cases
DLM operations using Command-Line Interface
Metadata replication
Replication policy operations
Cloud credentials operations
Miscellaneous
- Update Cluster Endpoint
Failing Over Manually
- Make the destination cluster the new source
  - Remove the Ranger deny policy
- Activate a new destination cluster
DLM version Information
Tuning DLM Engine
Troubleshooting DLM

Hive replication concepts

DLM supports Hive replication.

Hive tables - Managed and External
Managed tables are Hive owned tables where the entire lifecycle of the tables’ data are managed and controlled by Hive. External tables are tables where Hive has loose coupling with the data.
ACID tables replication
Hive managed tables supporting insert, update, delete operations with ACID semantics are called ACID tables. ACID tables support only ORC file format.
Bootstrap and incremental replication
DLM allows you to replicate Hive databases from a source cluster to a target location on a destination cluster.
Storage-based authorization
Hive supports doAs=true plus storage-based authorization that enables security at Hive Metastore API level.
Statistics replication
Basic statistics such as the number of rows of a table or partition and the column statistics such as histograms (min, max, count) of a particular interesting column are important in many ways.
Replication differences between HDP 2.6.5 to 3.x
In HDP 2.6.5 version, managed tables are managed by Hive.
Hive on-premise replication
Before you can begin replicating data using clusters on Hive, you must make sure that there are at least a couple of clusters that are registered in your DLM App instance. The replication load happens on the target cluster.

Parent topic: Replication of HIVE data

© 2012–2019, Hortonworks, Inc.

Document licensed under the Creative Commons Attribution ShareAlike 4.0 License.

Hortonworks.com | Documentation | Support | Community