cascading.tap
Class MultiSourceTap

java.lang.Object
  cascading.tap.Tap
      cascading.tap.SourceTap
          cascading.tap.MultiSourceTap

All Implemented Interfaces:

FlowElement, CompositeTap, Serializable

Direct Known Subclasses:

GlobHfs, MultiTap

public class MultiSourceTap
extends SourceTap
implements CompositeTap

Class MultiSourceTap is used to tie multiple Tap instances into a single resource. Effectively this will allow multiple files to be concatenated into the requesting pipe assembly, if they all share the same Scheme instance.

Note that order is not maintained by virtue of the underlying model. If order is necessary, use a unique sequence key to span the resources, like a line number.

Note that if multiple input files have the same Scheme (like TextLine), they may not contain the same semi-structure internally. For example, one file might be an Apache log file, and anoter might be a Log4J log file. If each one should be parsed differently, then they must be handled by different pipe assembly branches.

See Also:

Serialized Form

Field Summary

protected Tap[]

taps

Constructor Summary

protected	MultiSourceTap(Scheme scheme)
	MultiSourceTap(Tap... taps) Constructor MultiSourceTap creates a new MultiSourceTap instance.

Method Summary

boolean	equals(Object object)
Tap[]	getChildTaps()
Path	getPath() Method getPath() always returns null.
long	getPathModified(JobConf conf) Returns the most current modified time.
Scheme	getScheme() Method getScheme returns the scheme of this Tap object.
protected Tap[]	getTaps() Method getTaps returns the taps of this MultiTap object.
int	hashCode()
boolean	isReplace() Method isReplace indicates whether the resource represented by this instance should be deleted if it already exists when the Flow is started.
TupleEntryIterator	openForRead(JobConf conf) Method openForRead opens the resource represented by this Tap instance.
boolean	pathExists(JobConf conf) Method pathExists return true if the path represented by this instance exists.
void	sourceInit(JobConf conf) Method sourceInit initializes this instance as a source.
String	toString()

Methods inherited from class cascading.tap.SourceTap

deletePath, getSinkFields, isSink, makeDirs, openForWrite, sink

Methods inherited from class cascading.tap.Tap

flowInit, getIdentifier, getQualifiedPath, getSinkMode, getSourceFields, isAppend, isEquivalentTo, isKeep, isSource, isUpdate, isWriteDirect, outgoingScopeFor, resolveFields, resolveIncomingOperationFields, setScheme, setWriteDirect, sinkInit, source, taps

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

taps

protected Tap[] taps

Constructor Detail

MultiSourceTap

protected MultiSourceTap(Scheme scheme)

MultiSourceTap

@ConstructorProperties(value="taps")
public MultiSourceTap(Tap... taps)

Constructor MultiSourceTap creates a new MultiSourceTap instance.

Parameters:

taps - of type Tap...

Method Detail

getTaps

protected Tap[] getTaps()

Method getTaps returns the taps of this MultiTap object.

Returns:

the taps (type Tap[]) of this MultiTap object.

getChildTaps

public Tap[] getChildTaps()

Specified by:

getChildTaps in interface CompositeTap

getPath

public Path getPath()

Method getPath() always returns null. Since this class represents multiple resources, this is not one single path.

Specified by:

getPath in class Tap

Returns:

Path

getScheme

public Scheme getScheme()

Description copied from class: Tap

Method getScheme returns the scheme of this Tap object.

Overrides:

getScheme in class Tap

Returns:

the scheme (type Scheme) of this Tap object.

isReplace

public boolean isReplace()

Description copied from class: Tap

Method isReplace indicates whether the resource represented by this instance should be deleted if it already exists when the Flow is started.

Overrides:

isReplace in class Tap

Returns:

boolean

sourceInit

public void sourceInit(JobConf conf)
                throws IOException

Description copied from class: Tap

Method sourceInit initializes this instance as a source.

This method maybe called more than once if this Tap instance is used outside the scope of a Flow instance or if it participates in multiple times in a given Flow or across different Flows in a Cascade.

In the context of a Flow, it will be called after FlowListener.onStarting(cascading.flow.Flow)

Overrides:

sourceInit in class Tap

Parameters:

conf - of type JobConf

Throws:

IOException - on resource initialization failure.

pathExists

public boolean pathExists(JobConf conf)
                   throws IOException

Description copied from class: Tap

Method pathExists return true if the path represented by this instance exists.

Specified by:

pathExists in class Tap

Parameters:

conf - of type JobConf

Returns:

boolean

Throws:

IOException - when the status cannot be determined

getPathModified

public long getPathModified(JobConf conf)
                     throws IOException

Returns the most current modified time.

Specified by:

getPathModified in class Tap

Parameters:

conf - of type JobConf

Returns:

long

Throws:

IOException - when the modified date cannot be determined

openForRead

public TupleEntryIterator openForRead(JobConf conf)
                               throws IOException

Description copied from class: Tap

Method openForRead opens the resource represented by this Tap instance.

Overrides:

openForRead in class SourceTap

Parameters:

conf - of type JobConf

Returns:

TupleEntryIterator

Throws:

IOException - when the resource cannot be opened

equals

public boolean equals(Object object)

Overrides:

equals in class Tap

hashCode

public int hashCode()

Overrides:

hashCode in class Tap

toString

public String toString()

Overrides:

toString in class Object