public class DistCacheTap extends cascading.tap.hadoop.BaseDistCacheTap
DistributedCache
on read when accessed cluster side.
This is useful for HashJoin
s.
The distributed cache is only used when the Tap is used as a source. If the DistCacheTap is used as a sink,
it will delegate to the provided parent instance and not use the DistributedCache.metaInfo, original
Constructor and Description |
---|
DistCacheTap(cascading.tap.hadoop.Hfs parent)
Constructs a new DistCacheTap instance with the given Hfs.
|
Modifier and Type | Method and Description |
---|---|
protected void |
addLocalCacheFiles(Configuration conf,
URI uri) |
protected Path[] |
getLocalCacheFiles(FlowProcess<? extends Configuration> flowProcess) |
openForRead, sourceConfInit
commitResource, createResource, createResource, deleteResource, deleteResource, flowConfInit, getConfigDef, getFullIdentifier, getFullIdentifier, getIdentifier, getMetaInfo, getModifiedTime, getModifiedTime, getNodeConfigDef, getOriginal, getScheme, getSinkFields, getSinkMode, getSourceFields, getStepConfigDef, getTrace, hasConfigDef, hasNodeConfigDef, hasStepConfigDef, isKeep, isReplace, isSink, isSource, isTemporary, isUpdate, openForRead, openForWrite, openForWrite, outgoingScopeFor, prepareResourceForRead, prepareResourceForWrite, presentSinkFields, presentSourceFields, resolveIncomingOperationArgumentFields, resolveIncomingOperationPassThroughFields, resourceExists, resourceExists, retrieveSinkFields, retrieveSourceFields, rollbackResource, setMetaInfo, setOriginal, sinkConfInit, toString
public DistCacheTap(cascading.tap.hadoop.Hfs parent)
parent
- an Hfs or GlobHfs instance representing a small file.protected Path[] getLocalCacheFiles(FlowProcess<? extends Configuration> flowProcess) throws IOException
getLocalCacheFiles
in class cascading.tap.hadoop.BaseDistCacheTap
IOException
protected void addLocalCacheFiles(Configuration conf, URI uri)
addLocalCacheFiles
in class cascading.tap.hadoop.BaseDistCacheTap
Copyright © 2007-2015 Concurrent, Inc. All Rights Reserved.