cascading.tap.hadoop
Class DistCacheTap
java.lang.Object
cascading.tap.Tap<Config,Input,Output>
cascading.tap.DecoratorTap<Void,JobConf,RecordReader,OutputCollector>
cascading.tap.hadoop.DistCacheTap
- All Implemented Interfaces:
- FlowElement, Traceable, Serializable
public class DistCacheTap
- extends DecoratorTap<Void,JobConf,RecordReader,OutputCollector>
Class DistCacheTap is a Tap decorator for Hfs and can be used to move a file to the
DistributedCache
on read when accessed cluster side.
This is useful for HashJoin
s.
The distributed cache is only used when the Tap is used as a source. If the DistCacheTap is used as a sink,
it will delegate to the provided parent instance and not use the DistributedCache.
- See Also:
- Serialized Form
Constructor Summary |
DistCacheTap(Hfs parent)
Constructs a new DistCacheTap instance with the given Hfs. |
Methods inherited from class cascading.tap.DecoratorTap |
commitResource, createResource, createResource, deleteResource, deleteResource, flowConfInit, getConfigDef, getFullIdentifier, getFullIdentifier, getIdentifier, getMetaInfo, getModifiedTime, getModifiedTime, getOriginal, getScheme, getSinkFields, getSinkMode, getSourceFields, getStepConfigDef, getTrace, hasConfigDef, hasStepConfigDef, isEquivalentTo, isKeep, isReplace, isSink, isSource, isTemporary, isUpdate, openForRead, openForWrite, openForWrite, outgoingScopeFor, presentSinkFields, presentSourceFields, resolveIncomingOperationArgumentFields, resolveIncomingOperationPassThroughFields, resourceExists, resourceExists, retrieveSinkFields, retrieveSourceFields, rollbackResource, setMetaInfo, setOriginal, sinkConfInit, toString |
DistCacheTap
public DistCacheTap(Hfs parent)
- Constructs a new DistCacheTap instance with the given Hfs.
- Parameters:
parent
- an Hfs or GlobHfs instance representing a small file.
sourceConfInit
public void sourceConfInit(FlowProcess<JobConf> process,
JobConf conf)
- Overrides:
sourceConfInit
in class DecoratorTap<Void,JobConf,RecordReader,OutputCollector>
openForRead
public TupleEntryIterator openForRead(FlowProcess<JobConf> flowProcess,
RecordReader input)
throws IOException
- Overrides:
openForRead
in class DecoratorTap<Void,JobConf,RecordReader,OutputCollector>
- Throws:
IOException
Copyright © 2007-2014 Concurrent, Inc. All Rights Reserved.