org.apache.hadoop.hbase.backup.impl

Class BackupManager

java.lang.Object

org.apache.hadoop.hbase.backup.impl.BackupManager

All Implemented Interfaces:

Closeable, AutoCloseable

@InterfaceAudience.Private
 @InterfaceStability.Evolving
public class BackupManager
extends Object
implements Closeable

Handles backup requests on server-side, creates backup context records in hbase:backup to keep track backup. The timestamps kept in hbase:backup table will be used for future incremental backup. Creates BackupContext and DispatchRequest.

Constructor Summary

Constructors

Constructor and Description
BackupManager(Configuration conf) Backup manager constructor.

Method Summary

Modifier and Type	Method and Description
void	addIncrementalBackupTableSet(Set<TableName> tables) Adds set of tables to overall incremental backup table set
void	close() Stop all the work of backup.
BackupInfo	createBackupInfo(String backupId, BackupType type, List<TableName> tableList, String targetRootDir, int workers, long bandwidth) Create a BackupContext based on input backup request.
static void	decorateMasterConfiguration(Configuration conf) This method modifies the master's configuration in order to inject backup-related features
static void	decorateRSConfiguration(Configuration conf) This method modifies the RS configuration in order to inject backup-related features
ArrayList<BackupManifest.BackupImage>	getAncestors(BackupInfo backupCtx) Get direct ancestors of the current backup.
ArrayList<BackupManifest.BackupImage>	getAncestors(BackupInfo backupContext, TableName table) Get the direct ancestors of this backup for one table involved.
protected BackupInfo	getBackupContext() Return backup context
ArrayList<BackupInfo>	getBackupHistory() Get all completed backup information (in desc order by time)
ArrayList<BackupInfo>	getBackupHistory(boolean completed)
Connection	getConnection()
Set<TableName>	getIncrementalBackupTableSet() Return the current tables covered by incremental backup.
Iterator<org.apache.hadoop.hbase.backup.impl.BackupSystemTable.WALItem>	getWALFilesFromBackupSystem() Get WAL files iterator
void	initialize() Start the backup manager service.
static boolean	isBackupEnabled(Configuration conf)
String	readBackupStartCode() Read the last backup start code (timestamp) of last successful backup.
HashMap<TableName,HashMap<String,Long>>	readLogTimestampMap() Read the timestamp for each region server log after the last successful backup.
Pair<Map<TableName,Map<String,Map<String,List<Pair<String,Boolean>>>>>,List<byte[]>>	readOrigBulkloadRows(List<TableName> tableList)
HashMap<String,Long>	readRegionServerLastLogRollResult() Get the RS log information after the last log roll from hbase:backup.
void	recordWALFiles(List<String> files) Saves list of WAL files after incremental backup operation.
void	removeOrigBulkLoadedRows(List<TableName> lst, List<byte[]> rows)
void	setBackupInfo(BackupInfo backupInfo)
void	updateBackupInfo(BackupInfo context) Updates status (state) of a backup session in a persistent store
void	writeBackupStartCode(Long startCode) Write the start code (timestamp) to hbase:backup.
void	writeBulkLoadedFiles(List<TableName> sTableList, Map<byte[],List<Path>>[] maps)
void	writeRegionServerLogTimestamp(Set<TableName> tables, HashMap<String,Long> newTimestamps) Write the current timestamps for each regionserver to hbase:backup after a successful full or incremental backup.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BackupManager

public BackupManager(Configuration conf)
              throws IOException

Backup manager constructor.

Parameters:

conf - configuration

Throws:

IOException - exception

Method Detail

getBackupContext

protected BackupInfo getBackupContext()

Return backup context

decorateMasterConfiguration

public static void decorateMasterConfiguration(Configuration conf)

This method modifies the master's configuration in order to inject backup-related features

Parameters:

conf - configuration

decorateRSConfiguration

public static void decorateRSConfiguration(Configuration conf)

This method modifies the RS configuration in order to inject backup-related features

Parameters:

conf - configuration

isBackupEnabled

public static boolean isBackupEnabled(Configuration conf)

close

public void close()

Stop all the work of backup.

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

createBackupInfo

public BackupInfo createBackupInfo(String backupId,
                                   BackupType type,
                                   List<TableName> tableList,
                                   String targetRootDir,
                                   int workers,
                                   long bandwidth)
                            throws BackupException

Create a BackupContext based on input backup request.

Parameters:

backupId - backup id

type - type

tablelist - table list

targetRootDir - root dir

snapshot - snapshot name

Returns:

BackupContext context

Throws:

BackupException - exception

initialize

public void initialize()
                throws IOException

Start the backup manager service.

Throws:

IOException - exception

setBackupInfo

public void setBackupInfo(BackupInfo backupInfo)

getAncestors

public ArrayList<BackupManifest.BackupImage> getAncestors(BackupInfo backupCtx)
                                                   throws IOException,
                                                          BackupException

Get direct ancestors of the current backup.

Parameters:

backupCtx - The backup context for the current backup

Returns:

The ancestors for the current backup

Throws:

IOException - exception

BackupException - exception

getAncestors

public ArrayList<BackupManifest.BackupImage> getAncestors(BackupInfo backupContext,
                                                          TableName table)
                                                   throws BackupException,
                                                          IOException

Get the direct ancestors of this backup for one table involved.

Parameters:

backupContext - backup context

table - table

Returns:

backupImages on the dependency list

Throws:

BackupException - exception

IOException - exception

updateBackupInfo

public void updateBackupInfo(BackupInfo context)
                      throws IOException

Updates status (state) of a backup session in a persistent store

Parameters:

context - context

Throws:

IOException - exception

readBackupStartCode

public String readBackupStartCode()
                           throws IOException

Read the last backup start code (timestamp) of last successful backup. Will return null if there is no startcode stored in hbase:backup or the value is of length 0. These two cases indicate there is no successful backup completed so far.

Returns:

the timestamp of a last successful backup

Throws:

IOException - exception

writeBackupStartCode

public void writeBackupStartCode(Long startCode)
                          throws IOException

Write the start code (timestamp) to hbase:backup. If passed in null, then write 0 byte.

Parameters:

startCode - start code

Throws:

IOException - exception

readOrigBulkloadRows

public Pair<Map<TableName,Map<String,Map<String,List<Pair<String,Boolean>>>>>,List<byte[]>> readOrigBulkloadRows(List<TableName> tableList)
                                                                                                          throws IOException

Throws:

IOException

removeOrigBulkLoadedRows

public void removeOrigBulkLoadedRows(List<TableName> lst,
                                     List<byte[]> rows)
                              throws IOException

Throws:

IOException

writeBulkLoadedFiles

public void writeBulkLoadedFiles(List<TableName> sTableList,
                                 Map<byte[],List<Path>>[] maps)
                          throws IOException

Throws:

IOException

readRegionServerLastLogRollResult

public HashMap<String,Long> readRegionServerLastLogRollResult()
                                                       throws IOException

Get the RS log information after the last log roll from hbase:backup.

Returns:

RS log info

Throws:

IOException - exception

getBackupHistory

public ArrayList<BackupInfo> getBackupHistory()
                                       throws IOException

Get all completed backup information (in desc order by time)

Returns:

history info of BackupCompleteData

Throws:

IOException - exception

getBackupHistory

public ArrayList<BackupInfo> getBackupHistory(boolean completed)
                                       throws IOException

Throws:

IOException

writeRegionServerLogTimestamp

public void writeRegionServerLogTimestamp(Set<TableName> tables,
                                          HashMap<String,Long> newTimestamps)
                                   throws IOException

Write the current timestamps for each regionserver to hbase:backup after a successful full or incremental backup. Each table may have a different set of log timestamps. The saved timestamp is of the last log file that was backed up already.

Parameters:

tables - tables

Throws:

IOException - exception

readLogTimestampMap

public HashMap<TableName,HashMap<String,Long>> readLogTimestampMap()
                                                            throws IOException

Read the timestamp for each region server log after the last successful backup. Each table has its own set of the timestamps.

Returns:

the timestamp for each region server. key: tableName value: RegionServer,PreviousTimeStamp

Throws:

IOException - exception

getIncrementalBackupTableSet

public Set<TableName> getIncrementalBackupTableSet()
                                            throws IOException

Return the current tables covered by incremental backup.

Returns:

set of tableNames

Throws:

IOException - exception

addIncrementalBackupTableSet

public void addIncrementalBackupTableSet(Set<TableName> tables)
                                  throws IOException

Adds set of tables to overall incremental backup table set

Parameters:

tables - tables

Throws:

IOException - exception

recordWALFiles

public void recordWALFiles(List<String> files)
                    throws IOException

Saves list of WAL files after incremental backup operation. These files will be stored until TTL expiration and are used by Backup Log Cleaner plugin to determine which WAL files can be safely purged.

Throws:

IOException

getWALFilesFromBackupSystem

public Iterator<org.apache.hadoop.hbase.backup.impl.BackupSystemTable.WALItem> getWALFilesFromBackupSystem()
                                                                                                    throws IOException

Get WAL files iterator

Returns:

WAL files iterator from hbase:backup

Throws:

IOException

getConnection

public Connection getConnection()