Uses of Class
org.apache.pig.LoadFunc

Packages that use LoadFunc

org.apache.hadoop.zebra.pig	Implementation of PIG Storer/Loader Interfaces
org.apache.pig	Public interfaces and classes for Pig.
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators
org.apache.pig.backend.hadoop.hbase
org.apache.pig.builtin	This package contains builtin Pig UDFs.
org.apache.pig.impl.builtin
org.apache.pig.impl.io
org.apache.pig.impl.util
org.apache.pig.newplan.logical.relational
org.apache.pig.piggybank.storage
org.apache.pig.piggybank.storage.apachelog
org.apache.pig.piggybank.storage.avro
org.apache.pig.piggybank.storage.partition

Uses of LoadFunc in org.apache.hadoop.zebra.pig

Subclasses of LoadFunc in org.apache.hadoop.zebra.pig

class

TableLoader Pig IndexableLoadFunc and Slicer for Zebra Table

Uses of LoadFunc in org.apache.pig

Subclasses of LoadFunc in org.apache.pig

class

FileInputLoadFunc This class provides an implementation of OrderedLoadFunc interface which can be optionally re-used by LoadFuncs that use FileInputFormat, by having this as a super class

Uses of LoadFunc in org.apache.pig.backend.hadoop.executionengine.mapReduceLayer

Subclasses of LoadFunc in org.apache.pig.backend.hadoop.executionengine.mapReduceLayer

class

MergeJoinIndexer Merge Join indexer is used to generate on the fly index for doing Merge Join efficiently.

Constructors in org.apache.pig.backend.hadoop.executionengine.mapReduceLayer with parameters of type LoadFunc

PigRecordReader(org.apache.hadoop.mapreduce.InputFormat inputformat, PigSplit pigSplit, LoadFunc loadFunc, org.apache.hadoop.mapreduce.TaskAttemptContext context)

Uses of LoadFunc in org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators

Methods in org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators that return LoadFunc

LoadFunc

POLoad.getLoadFunc()

Constructors in org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators with parameters of type LoadFunc

POLoad(OperatorKey k, LoadFunc lf)

Uses of LoadFunc in org.apache.pig.backend.hadoop.hbase

Subclasses of LoadFunc in org.apache.pig.backend.hadoop.hbase

class

HBaseStorage A HBase implementation of LoadFunc and StoreFunc.

Uses of LoadFunc in org.apache.pig.builtin

Subclasses of LoadFunc in org.apache.pig.builtin

class	BinStorage Load and store data in a binary format.
class	PigStorage A load function that parses a line of input into fields using a delimiter to set the fields.
class	TextLoader This load function simply creates a tuple for each line of text that has a single chararray field that contains the line of text.

Uses of LoadFunc in org.apache.pig.impl.builtin

Subclasses of LoadFunc in org.apache.pig.impl.builtin

class	DefaultIndexableLoader Used by MergeJoin .
class	PoissonSampleLoader See "Skewed Join sampler" in http://wiki.apache.org/pig/PigSampler
class	RandomSampleLoader A loader that samples the data.
class	SampleLoader Abstract class that specifies the interface for sample loaders

Fields in org.apache.pig.impl.builtin declared as LoadFunc

protected LoadFunc

SampleLoader.loader

Uses of LoadFunc in org.apache.pig.impl.io

Subclasses of LoadFunc in org.apache.pig.impl.io

class	InterStorage LOAD FUNCTION FOR PIG INTERNAL USE ONLY! This load function is used for storing intermediate data between MR jobs of a pig query.
class	ReadToEndLoader This is wrapper Loader which wraps a real LoadFunc underneath and allows to read a file completely starting a given split (indicated by a split index which is used to look in the List returned by the underlying InputFormat's getSplits() method).
class	TFileStorage LOAD FUNCTION FOR PIG INTERNAL USE ONLY! This load function is used for storing intermediate data between MR jobs of a pig query.

Methods in org.apache.pig.impl.io with parameters of type LoadFunc

DataBag

PigFile.load(LoadFunc lfunc, PigContext pigContext)

Constructors in org.apache.pig.impl.io with parameters of type LoadFunc

ReadToEndLoader(LoadFunc wrappedLoadFunc, org.apache.hadoop.conf.Configuration conf, String inputLocation, int splitIndex)

ReadToEndLoader(LoadFunc wrappedLoadFunc, org.apache.hadoop.conf.Configuration conf, String inputLocation, int[] toReadSplitIdxs)
This constructor takes an array of split indexes (toReadSplitIdxs) of the splits to be read.

Uses of LoadFunc in org.apache.pig.impl.util

Methods in org.apache.pig.impl.util with parameters of type LoadFunc

static ResourceSchema

Utils.getSchema(LoadFunc wrappedLoadFunc, String location, boolean checkExistence, org.apache.hadoop.mapreduce.Job job)

Uses of LoadFunc in org.apache.pig.newplan.logical.relational

Methods in org.apache.pig.newplan.logical.relational that return LoadFunc

LoadFunc

LOLoad.getLoadFunc()

Uses of LoadFunc in org.apache.pig.piggybank.storage

Subclasses of LoadFunc in org.apache.pig.piggybank.storage

class	AllLoader The AllLoader provides the ability to point pig at a folder that contains files in multiple formats e.g.
class	CSVExcelStorage CSV loading and storing with support for multi-line fields, and escaping of delimiters and double quotes within fields; uses CSV conventions of Excel 2007.
class	CSVLoader A load function based on PigStorage that implements part of the CSV "standard" This loader properly supports double-quoted fields that contain commas and other double-quotes escaped with backslashes.
class	HadoopJobHistoryLoader
class	HiveColumnarLoader Loader for Hive RC Columnar files. Supports the following types: * Hive Type Pig Type from DataType string CHARARRAY int INTEGER bigint or long LONG float float double DOUBLE boolean BOOLEAN byte BYTE array TUPLE map MAP Partitions The input paths are scanned by the loader for [partition name]=[value] patterns in the subdirectories. If detected these partitions are appended to the table schema. For example if you have the directory structure:
class	MyRegExLoader
class	PigStorageSchema This Load/Store Func reads/writes metafiles that allow the schema and aliases to be determined at load time, saving one from having to manually enter schemas for pig-generated datasets.
class	RegExLoader RegExLoader is an abstract class used to parse logs based on a regular expression.
class	SequenceFileLoader A Loader for Hadoop-Standard SequenceFiles.
class	XMLLoader The load function to load the XML file This implements the LoadFunc interface which is used to parse records from a dataset.

Methods in org.apache.pig.piggybank.storage that return LoadFunc

LoadFunc

AllLoader.AllReader.prepareLoadFuncForReading(PigSplit split)

Uses of LoadFunc in org.apache.pig.piggybank.storage.apachelog

Subclasses of LoadFunc in org.apache.pig.piggybank.storage.apachelog

class

CombinedLogLoader CombinedLogLoader is used to load logs based on Apache's combined log format, based on a format like LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined The log filename ends up being access_log from a line like CustomLog logs/combined_log combined Example: raw = LOAD 'combined_log' USING org.apache.pig.piggybank.storage.apachelog.CombinedLogLoader AS (remoteAddr, remoteLogname, user, time, method, uri, proto, status, bytes, referer, userAgent);

class

CommonLogLoader CommonLogLoader is used to load logs based on Apache's common log format, based on a format like LogFormat "%h %l %u %t \"%r\" %>s %b" common The log filename ends up being access_log from a line like CustomLog logs/access_log common Example: raw = LOAD 'access_log' USING org.apache.pig.piggybank.storage.apachelog.CommongLogLoader AS (remoteAddr, remoteLogname, user, time, method, uri, proto, bytes);

Uses of LoadFunc in org.apache.pig.piggybank.storage.avro

Subclasses of LoadFunc in org.apache.pig.piggybank.storage.avro

class

AvroStorage AvroStorage is used to load/store Avro data Document can be found here

Uses of LoadFunc in org.apache.pig.piggybank.storage.partition

Method parameters in org.apache.pig.piggybank.storage.partition with type arguments of type LoadFunc

List<org.apache.hadoop.fs.FileStatus>	PathPartitionHelper.listStatus(org.apache.hadoop.mapreduce.JobContext ctx, Class<? extends LoadFunc> loaderClass, String signature) This method is called by the FileInputFormat to find the input paths for which splits should be calculated. If applyDateRanges == true: Then the HiveRCDateSplitter is used to apply filtering on the input files. Else the default FileInputFormat listStatus method is used.
void	PathPartitionHelper.setPartitionFilterExpression(String partitionFilterExpression, Class<? extends LoadFunc> loaderClass, String signature) Sets the PARITITION_FILTER_EXPRESSION property in the UDFContext identified by the loaderClass.
void	PathPartitionHelper.setPartitionKeys(String location, org.apache.hadoop.conf.Configuration conf, Class<? extends LoadFunc> loaderClass, String signature) Reads the partition keys from the location i.e the base directory