Hortonworks Docs
»
Data Lifecycle Manager 1.2.0
»
DLM Administration
DLM Administration
Also available as:
Introduction
Purpose and scope
Audience and assumptions
Replication concepts
Data Lifecycle Manager terminology
Communication with HDP clusters
How Policies Work in Data Lifecycle Manager
UI overview
Cluster Health panel
Policies panel
Jobs panel
Recent Issues panel
Clusters map
Issues & Updates table
Preparing to setup replication policy
Roles required
Infrastructure Admin role
DataPlane Admin role
Roles Required for Installation and Troubleshooting
Data Lifecycle Manager Tasks and Required Roles
Add clusters
Cluster pairing
Pairing considerations
Cloud credentials
Register cloud credentials
Registering Amazon S3 cloud account
Considerations for Amazon S3
Registering WASB cloud account
Considerations for WASB
Data replication use cases
Replication of data using HDFS
HDFS cloud replication
On-premise to on-premise replication in HDFS
Replication of data on-premise to on-premise in HDFS
On-premise to Amazon S3 replication in HDFS
Replication of data on-premise to Amazon S3 in HDFS
Amazon S3 to on-premise replication in HDFS
Replication of data from Amazon S3 to on-premise in HDFS
On-premise to WASB replication in HDFS
Replication of data on-premise to WASB in HDFS
WASB to on-premise replication in HDFS
Replication of data from WASB to on-premise in HDFS
Replication of data using Hive
Hive cloud replication
Hive replication bootstrap
Non-support of replication of Hive-Managed tables written by Spark applications.
On-premise to on-premise replication in Hive
Replication of data on-premise to on-premise in Hive
On-premise to Amazon S3 replication in Hive
Target cluster setup for Amazon S3
Replication of data on-premise to Amazon S3 in Hive
On-premise to WASB replication in Hive
Target cluster setup for WASB
Replication of data on-premise to WASB in Hive
Metadata replication
Ranger metadata
Atlas metadata
Snapshot replication between HDP clusters
Replication policy operations
Monitoring replication
Policies page
Overview page
Notifications page
Tuning replication policy (advanced options)
Update replication policy
Browsing data directory
Cloud credentials operations
Update cloud credentials
Delete credentials
Unregistered credentials
Miscellaneous
Update Cluster Endpoint
Failing Over Manually
Make the destination cluster the new source
Remove the Ranger deny policy
Activate a new destination cluster
DLM policy parameters
DLM version Information
Tuning DLM Engine
Troubleshooting DLM
Ranger UI does not display deny policy items
Hive cloud replication is slow
Replication fails with TDE and non-TDE data
Hive data cannot be replicated
Instance of a policy stuck in a running state
Hive replication failure
DLM out of memory
Replication of data using Hive
Hive cloud replication
DLM supports replication of the Hive database from a cluster with underlying HDFS to another cluster with cloud storage. It uses push-based replication, with the replication job running on the cluster with HDFS. Hive replication from cloud storage to HDFS is not supported.
Hive replication bootstrap
DLM allows you to replicate Hive databases from a source cluster to a target location on a destination cluster.
Non-support of replication of Hive-Managed tables written by Spark applications.
DLM Hive replication for Managed tables relies on replication events being published by Hive in Hive Metastore for every change that is made by Hive.
On-premise to on-premise replication in Hive
Before you can begin replicating data using clusters on Hive, you must make sure that there are at least a couple of clusters that are registered in your DLM App instance. The replication load happens on the target cluster.
On-premise to Amazon S3 replication in Hive
The process for creating a Hive data replication job from on-premise to Amazon S3 is similar to creating one for on-premise to on-premise. The primary difference is that, you must register your cloud account credentials with DLM App instance, so that DLM can access your cloud storage. The replication load happens on the source cluster.
On-premise to WASB replication in Hive
The process for creating a Hive data replication job from on-premise to WASB is similar to creating one for on-premise to on-premise. The primary difference is that, you must register your WASB cloud credentials with DLM App instance, so that DLM can access your WASB cloud storage.
Parent topic:
Data replication use cases
© 2012–2019, Hortonworks, Inc.
Document licensed under the
Creative Commons Attribution ShareAlike 4.0 License
.
Hortonworks.com
|
Documentation
|
Support
|
Community
