org.apache.hadoop.hbase.mob.mapreduce

Class SweepJob

java.lang.Object

org.apache.hadoop.hbase.mob.mapreduce.SweepJob

@InterfaceAudience.Private
public class SweepJob
extends Object

The sweep job. Run map reduce to merge the smaller mob files into bigger ones and cleans the unused ones.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SweepJob.DummyMobAbortable
static class	SweepJob.SweepCounter The counter used in sweep job.

Field Summary

Fields

Modifier and Type	Field and Description
static String	CREDENTIALS_LOCATION
static String	MOB_SWEEP_JOB_DELAY
protected static long	ONE_DAY
static String	WORKING_FILES_DIR_KEY

Constructor Summary

Constructors

Constructor and Description
SweepJob(Configuration conf, FileSystem fs)

Method Summary

Modifier and Type	Method and Description
int	sweep(TableName tn, HColumnDescriptor family) Runs MapReduce to do the sweeping on the mob files.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

WORKING_FILES_DIR_KEY

public static final String WORKING_FILES_DIR_KEY

See Also:

Constant Field Values

MOB_SWEEP_JOB_DELAY

public static final String MOB_SWEEP_JOB_DELAY

See Also:

Constant Field Values

ONE_DAY

protected static final long ONE_DAY

See Also:

Constant Field Values

CREDENTIALS_LOCATION

public static final String CREDENTIALS_LOCATION

See Also:

Constant Field Values

Constructor Detail

SweepJob

public SweepJob(Configuration conf,
                FileSystem fs)

Method Detail

sweep

public int sweep(TableName tn,
                 HColumnDescriptor family)
          throws IOException,
                 ClassNotFoundException,
                 InterruptedException,
                 KeeperException

Runs MapReduce to do the sweeping on the mob files. There's a MobReferenceOnlyFilter so that the mappers only get the cells that have mob references from 'normal' regions' rows. The running of the sweep tool on the same column family are mutually exclusive. The HBase major compaction and running of the sweep tool on the same column family are mutually exclusive. The synchronization is done by the Zookeeper. So in the beginning of the running, we need to make sure only this sweep tool is the only one that is currently running in this column family, and in this column family there're no major compaction in progress.

Parameters:

tn - The current table name.

family - The descriptor of the current column family.

Returns:

0 upon success, 3 if bailing out because another compaction is currently happening, or 4 the mr job was unsuccessful

Throws:

IOException

ClassNotFoundException

InterruptedException

KeeperException