org.apache.hadoop.hbase.wal

Class DefaultWALProvider

java.lang.Object

org.apache.hadoop.hbase.wal.DefaultWALProvider

All Implemented Interfaces:

WALProvider

@InterfaceAudience.Private
 @InterfaceStability.Evolving
public class DefaultWALProvider
extends Object
implements WALProvider

A WAL Provider that returns a single thread safe WAL that writes to Hadoop FS. By default, this implementation picks a directory in Hadoop FS based on a combination of

• the HBase root WAL directory
• HConstants.HREGION_LOGDIR_NAME
• the given factory's factoryId (usually identifying the regionserver by host:port)

It also uses the providerId to diffentiate among files.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	DefaultWALProvider.Reader
static interface	DefaultWALProvider.Writer

Field Summary

Fields

Modifier and Type	Field and Description
protected FSHLog	log
static String	META_WAL_PROVIDER_ID The hbase:meta region's WAL filename extension
static String	SPLITTING_EXT File Extension used while splitting an WAL into regions (HBASE-2312)
static String	WAL_FILE_NAME_DELIMITER

Constructor Summary

Constructors

Constructor and Description
DefaultWALProvider()

Method Summary

Modifier and Type	Method and Description
void	close() shutdown utstanding WALs and clean up any persisted state.
static DefaultWALProvider.Writer	createWriter(Configuration conf, FileSystem fs, Path path, boolean overwritable) public because of FSHLog.
static long	extractFileNumFromWAL(Path walName)
static long	extractFileNumFromWAL(WAL wal) It returns the file create timestamp from the file name.
static Path	getCurrentFileName(WAL wal) return the current filename from the current wal.
static long	getLogFileSize(WALFactory walFactory) iff the given WALFactory is using the DefaultWALProvider for meta and/or non-meta, count the size of files (rolled and active).
static long	getNumLogFiles(WALFactory walFactory) iff the given WALFactory is using the DefaultWALProvider for meta and/or non-meta, count the number of files (rolled and active).
static int	getNumRolledLogFiles(WAL wal) returns the number of rolled WAL files.
static ServerName	getServerNameFromWALDirectoryName(Configuration conf, String path) Pulls a ServerName out of a Path generated according to our layout rules.
static ServerName	getServerNameFromWALDirectoryName(Path logFile) This function returns region server name from a log file name which is in one of the following formats: hdfs:///hbase/.logs/-splitting/...
WAL	getWAL(byte[] identifier)
static String	getWALDirectoryName(String serverName) Construct the directory name for all WALs on a given server.
void	init(WALFactory factory, Configuration conf, List<WALActionsListener> listeners, String providerId) Set up the provider to create wals.
static boolean	isMetaFile(Path p)
void	shutdown() persist outstanding WALs to storage and stop accepting new appends.
static boolean	validateWALFilename(String filename) A WAL file name is of the format: <wal-name>WAL_FILE_NAME_DELIMITER<file-creation-timestamp>[.meta].

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

protected FSHLog log

WAL_FILE_NAME_DELIMITER

public static final String WAL_FILE_NAME_DELIMITER

See Also:

Constant Field Values

META_WAL_PROVIDER_ID

public static final String META_WAL_PROVIDER_ID

The hbase:meta region's WAL filename extension

See Also:

Constant Field Values

SPLITTING_EXT

public static final String SPLITTING_EXT

File Extension used while splitting an WAL into regions (HBASE-2312)

See Also:

Constant Field Values

Constructor Detail

DefaultWALProvider

public DefaultWALProvider()

Method Detail

init

public void init(WALFactory factory,
                 Configuration conf,
                 List<WALActionsListener> listeners,
                 String providerId)
          throws IOException

Description copied from interface: WALProvider

Set up the provider to create wals. will only be called once per instance.

Specified by:

init in interface WALProvider

Parameters:

factory - factory that made us, identity used for FS layout. may not be null

conf - may not be null

listeners - may be null

providerId - differentiate between providers from one facotry, used for FS layout. may be null

Throws:

IOException

getWAL

public WAL getWAL(byte[] identifier)
           throws IOException

Specified by:

getWAL in interface WALProvider

Parameters:

identifier - may not be null. contents will not be altered.

Returns:

a WAL for writing entries for the given region.

Throws:

IOException

close

public void close()
           throws IOException

Description copied from interface: WALProvider

shutdown utstanding WALs and clean up any persisted state. Call this method only when you will not need to replay any of the edits to the WALs from this provider. After this call completes, the underlying resources should have been reclaimed.

Specified by:

close in interface WALProvider

Throws:

IOException

shutdown

public void shutdown()
              throws IOException

Description copied from interface: WALProvider

persist outstanding WALs to storage and stop accepting new appends. This method serves as shorthand for sending a sync to every WAL provided by a given implementation. Those WALs will also stop accepting new writes.

Specified by:

shutdown in interface WALProvider

Throws:

IOException

getNumLogFiles

public static long getNumLogFiles(WALFactory walFactory)

iff the given WALFactory is using the DefaultWALProvider for meta and/or non-meta, count the number of files (rolled and active). if either of them aren't, count 0 for that provider.

Parameters:

walFactory - may not be null.

getLogFileSize

public static long getLogFileSize(WALFactory walFactory)

iff the given WALFactory is using the DefaultWALProvider for meta and/or non-meta, count the size of files (rolled and active). if either of them aren't, count 0 for that provider.

Parameters:

walFactory - may not be null.

getNumRolledLogFiles

public static int getNumRolledLogFiles(WAL wal)

returns the number of rolled WAL files.

getCurrentFileName

public static Path getCurrentFileName(WAL wal)

return the current filename from the current wal.

extractFileNumFromWAL

public static long extractFileNumFromWAL(WAL wal)

It returns the file create timestamp from the file name. For name format see validateWALFilename(String) public until remaining tests move to o.a.h.h.wal

Parameters:

wal - must not be null

Returns:

the file number that is part of the WAL file name

extractFileNumFromWAL

public static long extractFileNumFromWAL(Path walName)

validateWALFilename

public static boolean validateWALFilename(String filename)

A WAL file name is of the format: <wal-name>WAL_FILE_NAME_DELIMITER<file-creation-timestamp>[.meta]. provider-name is usually made up of a server-name and a provider-id

Parameters:

filename - name of the file to validate

Returns:

true if the filename matches an WAL, false otherwise

getWALDirectoryName

public static String getWALDirectoryName(String serverName)

Construct the directory name for all WALs on a given server.

Parameters:

serverName - Server name formatted as described in ServerName

Returns:

the relative WAL directory name, e.g. .logs/1.example.org,60030,12345 if serverName passed is 1.example.org,60030,12345

getServerNameFromWALDirectoryName

public static ServerName getServerNameFromWALDirectoryName(Configuration conf,
                                                           String path)
                                                    throws IOException

Pulls a ServerName out of a Path generated according to our layout rules. In the below layouts, this method ignores the format of the logfile component. Current format: [base directory for hbase]/hbase/.logs/ServerName/logfile or [base directory for hbase]/hbase/.logs/ServerName-splitting/logfile Expected to work for individual log files and server-specific directories.

Returns:

null if it's not a log file. Returns the ServerName of the region server that created this log file otherwise.

Throws:

IOException

getServerNameFromWALDirectoryName

public static ServerName getServerNameFromWALDirectoryName(Path logFile)

This function returns region server name from a log file name which is in one of the following formats:

• hdfs:///hbase/.logs/-splitting/...
• hdfs:///hbase/.logs//...

Parameters:

logFile -

Returns:

null if the passed in logFile isn't a valid WAL file path

isMetaFile

public static boolean isMetaFile(Path p)

createWriter

public static DefaultWALProvider.Writer createWriter(Configuration conf,
                                                     FileSystem fs,
                                                     Path path,
                                                     boolean overwritable)
                                              throws IOException

public because of FSHLog. Should be package-private

Throws:

IOException