org.apache.hadoop.hbase.backup.impl

Class BackupManager

java.lang.Object

org.apache.hadoop.hbase.backup.impl.BackupManager

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable

@InterfaceAudience.Private
 @InterfaceStability.Evolving
public class BackupManager
extends java.lang.Object
implements java.io.Closeable

Handles backup requests on server-side, creates backup context records in hbase:backup to keep track backup. The timestamps kept in hbase:backup table will be used for future incremental backup. Creates BackupContext and DispatchRequest.

Constructor Summary

Constructors

Constructor and Description
BackupManager(Configuration conf) Backup manager constructor.

Method Summary

Modifier and Type	Method and Description
void	addIncrementalBackupTableSet(java.util.Set<TableName> tables) Adds set of tables to overall incremental backup table set
void	close() Stop all the work of backup.
BackupInfo	createBackupInfo(java.lang.String backupId, BackupType type, java.util.List<TableName> tableList, java.lang.String targetRootDir, int workers, long bandwidth) Create a BackupContext based on input backup request.
static void	decorateMasterConfiguration(Configuration conf) This method modifies the master's configuration in order to inject backup-related features
static void	decorateRSConfiguration(Configuration conf) This method modifies the RS configuration in order to inject backup-related features
java.util.ArrayList<BackupManifest.BackupImage>	getAncestors(BackupInfo backupCtx) Get direct ancestors of the current backup.
java.util.ArrayList<BackupManifest.BackupImage>	getAncestors(BackupInfo backupContext, TableName table) Get the direct ancestors of this backup for one table involved.
protected BackupInfo	getBackupContext() Return backup context
java.util.ArrayList<BackupInfo>	getBackupHistory() Get all completed backup information (in desc order by time)
java.util.ArrayList<BackupInfo>	getBackupHistory(boolean completed)
Connection	getConnection()
java.util.Set<TableName>	getIncrementalBackupTableSet() Return the current tables covered by incremental backup.
java.util.Iterator<org.apache.hadoop.hbase.backup.impl.BackupSystemTable.WALItem>	getWALFilesFromBackupSystem() Get WAL files iterator
void	initialize() Start the backup manager service.
static boolean	isBackupEnabled(Configuration conf)
java.lang.String	readBackupStartCode() Read the last backup start code (timestamp) of last successful backup.
java.util.HashMap<TableName,java.util.HashMap<java.lang.String,java.lang.Long>>	readLogTimestampMap() Read the timestamp for each region server log after the last successful backup.
Pair<java.util.Map<TableName,java.util.Map<java.lang.String,java.util.Map<java.lang.String,java.util.List<Pair<java.lang.String,java.lang.Boolean>>>>>,java.util.List<byte[]>>	readOrigBulkloadRows(java.util.List<TableName> tableList)
java.util.HashMap<java.lang.String,java.lang.Long>	readRegionServerLastLogRollResult() Get the RS log information after the last log roll from hbase:backup.
void	recordWALFiles(java.util.List<java.lang.String> files) Saves list of WAL files after incremental backup operation.
void	removeOrigBulkLoadedRows(java.util.List<TableName> lst, java.util.List<byte[]> rows)
void	setBackupInfo(BackupInfo backupInfo)
void	updateBackupInfo(BackupInfo context) Updates status (state) of a backup session in a persistent store
void	writeBackupStartCode(java.lang.Long startCode) Write the start code (timestamp) to hbase:backup.
void	writeBulkLoadedFiles(java.util.List<TableName> sTableList, java.util.Map<byte[],java.util.List<Path>>[] maps)
void	writeRegionServerLogTimestamp(java.util.Set<TableName> tables, java.util.HashMap<java.lang.String,java.lang.Long> newTimestamps) Write the current timestamps for each regionserver to hbase:backup after a successful full or incremental backup.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BackupManager

public BackupManager(Configuration conf)
              throws java.io.IOException

Backup manager constructor.

Parameters:

conf - configuration

Throws:

java.io.IOException - exception

Method Detail

getBackupContext

protected BackupInfo getBackupContext()

Return backup context

decorateMasterConfiguration

public static void decorateMasterConfiguration(Configuration conf)

This method modifies the master's configuration in order to inject backup-related features

Parameters:

conf - configuration

decorateRSConfiguration

public static void decorateRSConfiguration(Configuration conf)

This method modifies the RS configuration in order to inject backup-related features

Parameters:

conf - configuration

isBackupEnabled

public static boolean isBackupEnabled(Configuration conf)

close

public void close()

Stop all the work of backup.

Specified by:

close in interface java.io.Closeable

Specified by:

close in interface java.lang.AutoCloseable

createBackupInfo

public BackupInfo createBackupInfo(java.lang.String backupId,
                                   BackupType type,
                                   java.util.List<TableName> tableList,
                                   java.lang.String targetRootDir,
                                   int workers,
                                   long bandwidth)
                            throws BackupException

Create a BackupContext based on input backup request.

Parameters:

backupId - backup id

type - type

tablelist - table list

targetRootDir - root dir

snapshot - snapshot name

Returns:

BackupContext context

Throws:

BackupException - exception

initialize

public void initialize()
                throws java.io.IOException

Start the backup manager service.

Throws:

java.io.IOException - exception

setBackupInfo

public void setBackupInfo(BackupInfo backupInfo)

getAncestors

public java.util.ArrayList<BackupManifest.BackupImage> getAncestors(BackupInfo backupCtx)
                                                             throws java.io.IOException,
                                                                    BackupException

Get direct ancestors of the current backup.

Parameters:

backupCtx - The backup context for the current backup

Returns:

The ancestors for the current backup

Throws:

java.io.IOException - exception

BackupException - exception

getAncestors

public java.util.ArrayList<BackupManifest.BackupImage> getAncestors(BackupInfo backupContext,
                                                                    TableName table)
                                                             throws BackupException,
                                                                    java.io.IOException

Get the direct ancestors of this backup for one table involved.

Parameters:

backupContext - backup context

table - table

Returns:

backupImages on the dependency list

Throws:

BackupException - exception

java.io.IOException - exception

updateBackupInfo

public void updateBackupInfo(BackupInfo context)
                      throws java.io.IOException

Updates status (state) of a backup session in a persistent store

Parameters:

context - context

Throws:

java.io.IOException - exception

readBackupStartCode

public java.lang.String readBackupStartCode()
                                     throws java.io.IOException

Read the last backup start code (timestamp) of last successful backup. Will return null if there is no startcode stored in hbase:backup or the value is of length 0. These two cases indicate there is no successful backup completed so far.

Returns:

the timestamp of a last successful backup

Throws:

java.io.IOException - exception

writeBackupStartCode

public void writeBackupStartCode(java.lang.Long startCode)
                          throws java.io.IOException

Write the start code (timestamp) to hbase:backup. If passed in null, then write 0 byte.

Parameters:

startCode - start code

Throws:

java.io.IOException - exception

readOrigBulkloadRows

public Pair<java.util.Map<TableName,java.util.Map<java.lang.String,java.util.Map<java.lang.String,java.util.List<Pair<java.lang.String,java.lang.Boolean>>>>>,java.util.List<byte[]>> readOrigBulkloadRows(java.util.List<TableName> tableList)
                                                                                                                                                                                                    throws java.io.IOException

Throws:

java.io.IOException

removeOrigBulkLoadedRows

public void removeOrigBulkLoadedRows(java.util.List<TableName> lst,
                                     java.util.List<byte[]> rows)
                              throws java.io.IOException

Throws:

java.io.IOException

writeBulkLoadedFiles

public void writeBulkLoadedFiles(java.util.List<TableName> sTableList,
                                 java.util.Map<byte[],java.util.List<Path>>[] maps)
                          throws java.io.IOException

Throws:

java.io.IOException

readRegionServerLastLogRollResult

public java.util.HashMap<java.lang.String,java.lang.Long> readRegionServerLastLogRollResult()
                                                                                     throws java.io.IOException

Get the RS log information after the last log roll from hbase:backup.

Returns:

RS log info

Throws:

java.io.IOException - exception

getBackupHistory

public java.util.ArrayList<BackupInfo> getBackupHistory()
                                                 throws java.io.IOException

Get all completed backup information (in desc order by time)

Returns:

history info of BackupCompleteData

Throws:

java.io.IOException - exception

getBackupHistory

public java.util.ArrayList<BackupInfo> getBackupHistory(boolean completed)
                                                 throws java.io.IOException

Throws:

java.io.IOException

writeRegionServerLogTimestamp

public void writeRegionServerLogTimestamp(java.util.Set<TableName> tables,
                                          java.util.HashMap<java.lang.String,java.lang.Long> newTimestamps)
                                   throws java.io.IOException

Write the current timestamps for each regionserver to hbase:backup after a successful full or incremental backup. Each table may have a different set of log timestamps. The saved timestamp is of the last log file that was backed up already.

Parameters:

tables - tables

Throws:

java.io.IOException - exception

readLogTimestampMap

public java.util.HashMap<TableName,java.util.HashMap<java.lang.String,java.lang.Long>> readLogTimestampMap()
                                                                                                    throws java.io.IOException

Read the timestamp for each region server log after the last successful backup. Each table has its own set of the timestamps.

Returns:

the timestamp for each region server. key: tableName value: RegionServer,PreviousTimeStamp

Throws:

java.io.IOException - exception

getIncrementalBackupTableSet

public java.util.Set<TableName> getIncrementalBackupTableSet()
                                                      throws java.io.IOException

Return the current tables covered by incremental backup.

Returns:

set of tableNames

Throws:

java.io.IOException - exception

addIncrementalBackupTableSet

public void addIncrementalBackupTableSet(java.util.Set<TableName> tables)
                                  throws java.io.IOException

Adds set of tables to overall incremental backup table set

Parameters:

tables - tables

Throws:

java.io.IOException - exception

recordWALFiles

public void recordWALFiles(java.util.List<java.lang.String> files)
                    throws java.io.IOException

Saves list of WAL files after incremental backup operation. These files will be stored until TTL expiration and are used by Backup Log Cleaner plugin to determine which WAL files can be safely purged.

Throws:

java.io.IOException

getWALFilesFromBackupSystem

public java.util.Iterator<org.apache.hadoop.hbase.backup.impl.BackupSystemTable.WALItem> getWALFilesFromBackupSystem()
                                                                                                              throws java.io.IOException

Get WAL files iterator

Returns:

WAL files iterator from hbase:backup

Throws:

java.io.IOException

getConnection

public Connection getConnection()