org.apache.hadoop.hbase.replication

Class ReplicationQueuesZKImpl

java.lang.Object

org.apache.hadoop.hbase.replication.ReplicationStateZKBase

org.apache.hadoop.hbase.replication.ReplicationQueuesZKImpl

All Implemented Interfaces:

ReplicationQueues

@InterfaceAudience.Private
public class ReplicationQueuesZKImpl
extends ReplicationStateZKBase
implements ReplicationQueues

This class provides an implementation of the ReplicationQueues interface using Zookeeper. The base znode that this class works at is the myQueuesZnode. The myQueuesZnode contains a list of all outstanding WAL files on this region server that need to be replicated. The myQueuesZnode is the regionserver name (a concatenation of the region server’s hostname, client port and start code). For example: /hbase/replication/rs/hostname.example.org,6020,1234 Within this znode, the region server maintains a set of WAL replication queues. These queues are represented by child znodes named using there give queue id. For example: /hbase/replication/rs/hostname.example.org,6020,1234/1 /hbase/replication/rs/hostname.example.org,6020,1234/2 Each queue has one child znode for every WAL that still needs to be replicated. The value of these WAL child znodes is the latest position that has been replicated. This position is updated every time a WAL entry is replicated. For example: /hbase/replication/rs/hostname.example.org,6020,1234/1/23522342.23422 [VALUE: 254]

Field Summary

Fields inherited from class org.apache.hadoop.hbase.replication.ReplicationStateZKBase

abortable, conf, DISABLED_ZNODE_BYTES, ENABLED_ZNODE_BYTES, hfileRefsZNode, ourClusterKey, peerStateNodeName, peersZNode, queuesZNode, replicationForBulkLoadEnabled, replicationZNode, zookeeper, ZOOKEEPER_ZNODE_REPLICATION_HFILE_REFS_DEFAULT, ZOOKEEPER_ZNODE_REPLICATION_HFILE_REFS_KEY

Constructor Summary

Constructors

Constructor and Description
ReplicationQueuesZKImpl(ZooKeeperWatcher zk, Configuration conf, Abortable abortable)

Method Summary

Modifier and Type	Method and Description
void	addHFileRefs(java.lang.String peerId, java.util.List<Pair<Path,Path>> pairs) Add new hfile references to the queue.
void	addLog(java.lang.String queueId, java.lang.String filename) Add a new WAL file to the given queue.
void	addPeerToHFileRefs(java.lang.String peerId) Add a peer to hfile reference queue if peer does not exist.
java.util.SortedMap<java.lang.String,java.util.SortedSet<java.lang.String>>	claimQueues(java.lang.String regionserverZnode) Take ownership for the set of queues belonging to a dead region server.
java.util.List<java.lang.String>	getAllQueues() Get a list of all queues for this region server.
long	getLogPosition(java.lang.String queueId, java.lang.String filename) Get the current position for a specific WAL in a given queue.
java.util.List<java.lang.String>	getLogsInQueue(java.lang.String queueId) Get a list of all WALs in the given queue.
void	init(java.lang.String serverName) Initialize the region server replication queue interface.
boolean	isThisOurZnode(java.lang.String znode) Checks if the provided znode is the same as this region server's
void	removeAllQueues() Remove all replication queues for this region server.
void	removeHFileRefs(java.lang.String peerId, java.util.List<java.lang.String> files) Remove hfile references from the queue.
void	removeLog(java.lang.String queueId, java.lang.String filename) Remove an WAL file from the given queue.
void	removeQueue(java.lang.String queueId) Remove a replication queue.
void	setLogPosition(java.lang.String queueId, java.lang.String filename, long position) Set the current position for a specific WAL in a given queue.

Methods inherited from class org.apache.hadoop.hbase.replication.ReplicationStateZKBase

getListOfReplicators, isPeerPath, peerExists, toByteArray

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.replication.ReplicationQueues

getListOfReplicators

Constructor Detail

ReplicationQueuesZKImpl

public ReplicationQueuesZKImpl(ZooKeeperWatcher zk,
                               Configuration conf,
                               Abortable abortable)

Method Detail

init

public void init(java.lang.String serverName)
          throws ReplicationException

Description copied from interface: ReplicationQueues

Initialize the region server replication queue interface.

Specified by:

init in interface ReplicationQueues

Parameters:

serverName - The server name of the region server that owns the replication queues this interface manages.

Throws:

ReplicationException

removeQueue

public void removeQueue(java.lang.String queueId)

Description copied from interface: ReplicationQueues

Remove a replication queue.

Specified by:

removeQueue in interface ReplicationQueues

Parameters:

queueId - a String that identifies the queue.

addLog

public void addLog(java.lang.String queueId,
                   java.lang.String filename)
            throws ReplicationException

Description copied from interface: ReplicationQueues

Add a new WAL file to the given queue. If the queue does not exist it is created.

Specified by:

addLog in interface ReplicationQueues

Parameters:

queueId - a String that identifies the queue.

filename - name of the WAL

Throws:

ReplicationException

removeLog

public void removeLog(java.lang.String queueId,
                      java.lang.String filename)

Description copied from interface: ReplicationQueues

Remove an WAL file from the given queue.

Specified by:

removeLog in interface ReplicationQueues

Parameters:

queueId - a String that identifies the queue.

filename - name of the WAL

setLogPosition

public void setLogPosition(java.lang.String queueId,
                           java.lang.String filename,
                           long position)

Description copied from interface: ReplicationQueues

Set the current position for a specific WAL in a given queue.

Specified by:

setLogPosition in interface ReplicationQueues

Parameters:

queueId - a String that identifies the queue

filename - name of the WAL

position - the current position in the file

getLogPosition

public long getLogPosition(java.lang.String queueId,
                           java.lang.String filename)
                    throws ReplicationException

Description copied from interface: ReplicationQueues

Get the current position for a specific WAL in a given queue.

Specified by:

getLogPosition in interface ReplicationQueues

Parameters:

queueId - a String that identifies the queue

filename - name of the WAL

Returns:

the current position in the file

Throws:

ReplicationException

isThisOurZnode

public boolean isThisOurZnode(java.lang.String znode)

Description copied from interface: ReplicationQueues

Checks if the provided znode is the same as this region server's

Specified by:

isThisOurZnode in interface ReplicationQueues

Parameters:

znode - to check

Returns:

if this is this rs's znode

claimQueues

public java.util.SortedMap<java.lang.String,java.util.SortedSet<java.lang.String>> claimQueues(java.lang.String regionserverZnode)

Description copied from interface: ReplicationQueues

Take ownership for the set of queues belonging to a dead region server.

Specified by:

claimQueues in interface ReplicationQueues

Parameters:

regionserverZnode - the id of the dead region server

Returns:

A SortedMap of the queues that have been claimed, including a SortedSet of WALs in each queue. Returns an empty map if no queues were failed-over.

removeAllQueues

public void removeAllQueues()

Description copied from interface: ReplicationQueues

Remove all replication queues for this region server.

Specified by:

removeAllQueues in interface ReplicationQueues

getLogsInQueue

public java.util.List<java.lang.String> getLogsInQueue(java.lang.String queueId)

Description copied from interface: ReplicationQueues

Get a list of all WALs in the given queue.

Specified by:

getLogsInQueue in interface ReplicationQueues

Parameters:

queueId - a String that identifies the queue

Returns:

a list of WALs, null if this region server is dead and has no outstanding queues

getAllQueues

public java.util.List<java.lang.String> getAllQueues()

Description copied from interface: ReplicationQueues

Get a list of all queues for this region server.

Specified by:

getAllQueues in interface ReplicationQueues

Returns:

a list of queueIds, null if this region server is dead and has no outstanding queues

addHFileRefs

public void addHFileRefs(java.lang.String peerId,
                         java.util.List<Pair<Path,Path>> pairs)
                  throws ReplicationException

Description copied from interface: ReplicationQueues

Add new hfile references to the queue.

Specified by:

addHFileRefs in interface ReplicationQueues

Parameters:

peerId - peer cluster id to which the hfiles need to be replicated

pairs - list of pairs of { HFile location in staging dir, HFile path in region dir which will be added in the queue }

Throws:

ReplicationException - if fails to add a hfile reference

removeHFileRefs

public void removeHFileRefs(java.lang.String peerId,
                            java.util.List<java.lang.String> files)

Description copied from interface: ReplicationQueues

Remove hfile references from the queue.

Specified by:

removeHFileRefs in interface ReplicationQueues

Parameters:

peerId - peer cluster id from which this hfile references needs to be removed

files - list of hfile references to be removed

addPeerToHFileRefs

public void addPeerToHFileRefs(java.lang.String peerId)
                        throws ReplicationException

Description copied from interface: ReplicationQueues

Add a peer to hfile reference queue if peer does not exist.

Specified by:

addPeerToHFileRefs in interface ReplicationQueues

Parameters:

peerId - peer cluster id to be added

Throws:

ReplicationException - if fails to add a peer id to hfile reference queue