cascading.tap.hadoop
Class MultiInputFormat

java.lang.Object
  cascading.tap.hadoop.MultiInputFormat

All Implemented Interfaces:

InputFormat

public class MultiInputFormat
extends Object
implements InputFormat

Class MultiInputFormat accepts multiple InputFormat class declarations allowing a single MR job to read data from incompatible file types.

Constructor Summary

MultiInputFormat()

Method Summary

static void	addInputFormat(JobConf toJob, JobConf... fromJobs) Used to set the current JobConf with all sub jobs configurations.
static Map<String,String>	getConfig(JobConf toJob, JobConf fromJob)
static JobConf[]	getJobConfs(JobConf job, List<Map<String,String>> configs)
RecordReader	getRecordReader(InputSplit split, JobConf job, Reporter reporter) Method getRecordReader delegates to the appropriate InputFormat.
InputSplit[]	getSplits(JobConf job, int numSplits) Method getSplits delegates to the appropriate InputFormat.
void	validateInput(JobConf job)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MultiInputFormat

public MultiInputFormat()

Method Detail

addInputFormat

public static void addInputFormat(JobConf toJob,
                                  JobConf... fromJobs)

Used to set the current JobConf with all sub jobs configurations.

Parameters:

toJob -

fromJobs -

getConfig

public static Map<String,String> getConfig(JobConf toJob,
                                           JobConf fromJob)

getJobConfs

public static JobConf[] getJobConfs(JobConf job,
                                    List<Map<String,String>> configs)

validateInput

public void validateInput(JobConf job)
                   throws IOException

Throws:

IOException

getSplits

public InputSplit[] getSplits(JobConf job,
                              int numSplits)
                       throws IOException

Method getSplits delegates to the appropriate InputFormat.

Specified by:

getSplits in interface InputFormat

Parameters:

job - of type JobConf

numSplits - of type int

Returns:

InputSplit[]

Throws:

IOException - when

getRecordReader

public RecordReader getRecordReader(InputSplit split,
                                    JobConf job,
                                    Reporter reporter)
                             throws IOException

Method getRecordReader delegates to the appropriate InputFormat.

Specified by:

getRecordReader in interface InputFormat

Parameters:

split - of type InputSplit

job - of type JobConf

reporter - of type Reporter

Returns:

RecordReader

Throws:

IOException - when