Class GlobHfs

  extended by cascading.tap.Tap
      extended by cascading.tap.SourceTap
          extended by cascading.tap.MultiSourceTap
              extended by cascading.tap.GlobHfs
All Implemented Interfaces:
FlowElement, CompositeTap, Serializable

public class GlobHfs
extends MultiSourceTap

Class GlobHfs is a type of MultiSourceTap that accepts Hadoop style 'file globbing' expressions so multiple files that match the given pattern may be used as the input sources for a given Flow.

See FileSystem.globStatus(org.apache.hadoop.fs.Path) for details on the globbing syntax. But in short it is similiar to standard regular expressions except alternation is done via {foo,bar} instead of (foo|bar).

Note that a Flow sourcing from GlobHfs is not currently compatible with the Cascade scheduler. GlobHfs expects the files and paths to exist so the wildcards can be resolved into concrete values so that the scheduler can order the Flows properly.

See Also:
Hfs, MultiSourceTap, FileSystem, Serialized Form

Field Summary
Fields inherited from class cascading.tap.MultiSourceTap
Constructor Summary
GlobHfs(Scheme scheme, String pathPattern)
          Constructor GlobHfs creates a new GlobHfs instance.
GlobHfs(Scheme scheme, String pathPattern, PathFilter pathFilter)
          Constructor GlobHfs creates a new GlobHfs instance.
Method Summary
 boolean equals(Object object)
protected  Tap[] getTaps()
          Method getTaps returns the taps of this MultiTap object.
 int hashCode()
 void sourceInit(JobConf conf)
          Method sourceInit initializes this instance as a source.
 String toString()
Methods inherited from class cascading.tap.MultiSourceTap
getChildTaps, getPath, getPathModified, getScheme, isReplace, openForRead, pathExists
Methods inherited from class cascading.tap.SourceTap
deletePath, getSinkFields, isSink, makeDirs, openForWrite, sink
Methods inherited from class cascading.tap.Tap
flowInit, getIdentifier, getQualifiedPath, getSinkMode, getSourceFields, isAppend, isEquivalentTo, isKeep, isSource, isUpdate, isWriteDirect, outgoingScopeFor, resolveFields, resolveIncomingOperationFields, setScheme, setWriteDirect, sinkInit, source, taps
Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail


public GlobHfs(Scheme scheme,
                                          String pathPattern)
Constructor GlobHfs creates a new GlobHfs instance.

scheme - of type Scheme
pathPattern - of type String


public GlobHfs(Scheme scheme,
                                          String pathPattern,
                                          PathFilter pathFilter)
Constructor GlobHfs creates a new GlobHfs instance.

scheme - of type Scheme
pathPattern - of type String
pathFilter - of type PathFilter
Method Detail


protected Tap[] getTaps()
Description copied from class: MultiSourceTap
Method getTaps returns the taps of this MultiTap object.

getTaps in class MultiSourceTap
the taps (type Tap[]) of this MultiTap object.


public void sourceInit(JobConf conf)
                throws IOException
Description copied from class: Tap
Method sourceInit initializes this instance as a source.

This method maybe called more than once if this Tap instance is used outside the scope of a Flow instance or if it participates in multiple times in a given Flow or across different Flows in a Cascade.

In the context of a Flow, it will be called after FlowListener.onStarting(cascading.flow.Flow)

sourceInit in class MultiSourceTap
conf - of type JobConf
IOException - on resource initialization failure.


public boolean equals(Object object)
equals in class MultiSourceTap


public int hashCode()
hashCode in class MultiSourceTap


public String toString()
toString in class MultiSourceTap

Copyright © 2007-2010 Concurrent, Inc. All Rights Reserved.