cascading.tap
Class GlobHfs

java.lang.Object
  cascading.tap.Tap
      cascading.tap.SourceTap
          cascading.tap.MultiSourceTap
              cascading.tap.GlobHfs

All Implemented Interfaces:

FlowElement, CompositeTap, Serializable

public class GlobHfs
extends MultiSourceTap

Class GlobHfs is a type of MultiSourceTap that accepts Hadoop style 'file globbing' expressions so multiple files that match the given pattern may be used as the input sources for a given Flow.

See FileSystem.globStatus(org.apache.hadoop.fs.Path) for details on the globbing syntax. But in short it is similiar to standard regular expressions except alternation is done via {foo,bar} instead of (foo|bar).

Note that a Flow sourcing from GlobHfs is not currently compatible with the Cascade scheduler. GlobHfs expects the files and paths to exist so the wildcards can be resolved into concrete values so that the scheduler can order the Flows properly.

See Also:

Hfs, MultiSourceTap, FileSystem, Serialized Form

Field Summary

Fields inherited from class cascading.tap.MultiSourceTap

taps

Constructor Summary

GlobHfs(Scheme scheme, String pathPattern)
Constructor GlobHfs creates a new GlobHfs instance.

GlobHfs(Scheme scheme, String pathPattern, PathFilter pathFilter)
Constructor GlobHfs creates a new GlobHfs instance.

Method Summary

boolean	equals(Object object)
protected Tap[]	getTaps() Method getTaps returns the taps of this MultiTap object.
int	hashCode()
void	sourceInit(JobConf conf) Method sourceInit initializes this instance as a source.
String	toString()

Methods inherited from class cascading.tap.MultiSourceTap

getChildTaps, getPath, getPathModified, getScheme, isReplace, openForRead, pathExists

Methods inherited from class cascading.tap.SourceTap

deletePath, getSinkFields, isSink, makeDirs, openForWrite, sink

Methods inherited from class cascading.tap.Tap

flowInit, getIdentifier, getQualifiedPath, getSinkMode, getSourceFields, isAppend, isEquivalentTo, isKeep, isSource, isUpdate, isWriteDirect, outgoingScopeFor, resolveFields, resolveIncomingOperationFields, setScheme, setWriteDirect, sinkInit, source, taps

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

GlobHfs

@ConstructorProperties(value={"scheme","pathPattern"})
public GlobHfs(Scheme scheme,
                                          String pathPattern)

Constructor GlobHfs creates a new GlobHfs instance.

Parameters:

scheme - of type Scheme

pathPattern - of type String

GlobHfs

@ConstructorProperties(value={"scheme","pathPattern","pathFilter"})
public GlobHfs(Scheme scheme,
                                          String pathPattern,
                                          PathFilter pathFilter)

Constructor GlobHfs creates a new GlobHfs instance.

Parameters:

scheme - of type Scheme

pathPattern - of type String

pathFilter - of type PathFilter

Method Detail

getTaps

protected Tap[] getTaps()

Description copied from class: MultiSourceTap

Method getTaps returns the taps of this MultiTap object.

Overrides:

getTaps in class MultiSourceTap

Returns:

the taps (type Tap[]) of this MultiTap object.

sourceInit

public void sourceInit(JobConf conf)
                throws IOException

Description copied from class: Tap

Method sourceInit initializes this instance as a source.

This method maybe called more than once if this Tap instance is used outside the scope of a Flow instance or if it participates in multiple times in a given Flow or across different Flows in a Cascade.

In the context of a Flow, it will be called after FlowListener.onStarting(cascading.flow.Flow)

Overrides:

sourceInit in class MultiSourceTap

Parameters:

conf - of type JobConf

Throws:

IOException - on resource initialization failure.

equals

public boolean equals(Object object)

Overrides:

equals in class MultiSourceTap

hashCode

public int hashCode()

Overrides:

hashCode in class MultiSourceTap

toString

public String toString()

Overrides:

toString in class MultiSourceTap