public class DistCacheTap extends cascading.tap.hadoop.BaseDistCacheTap
DistributedCache
on read when accessed cluster side.
This is useful for HashJoin
s.
The distributed cache is only used when the Tap is used as a source. If the DistCacheTap is used as a sink,
it will delegate to the provided parent instance and not use the DistributedCache.Constructor and Description |
---|
DistCacheTap(cascading.tap.hadoop.Hfs parent)
Constructs a new DistCacheTap instance with the given Hfs.
|
Modifier and Type | Method and Description |
---|---|
protected void |
addLocalCacheFiles(Configuration conf,
java.net.URI uri) |
protected Path[] |
getLocalCacheFiles(cascading.flow.FlowProcess<? extends Configuration> flowProcess) |
commitResource, createResource, createResource, deleteResource, deleteResource, flowConfInit, getConfigDef, getFullIdentifier, getFullIdentifier, getIdentifier, getMetaInfo, getModifiedTime, getModifiedTime, getNodeConfigDef, getOriginal, getScheme, getSinkFields, getSinkMode, getSourceFields, getStepConfigDef, getTrace, hasConfigDef, hasNodeConfigDef, hasStepConfigDef, isKeep, isReplace, isSink, isSource, isTemporary, isUpdate, openForRead, openForWrite, openForWrite, outgoingScopeFor, prepareResourceForRead, prepareResourceForWrite, presentSinkFields, presentSourceFields, resolveIncomingOperationArgumentFields, resolveIncomingOperationPassThroughFields, resourceExists, resourceExists, retrieveSinkFields, retrieveSourceFields, rollbackResource, setMetaInfo, setOriginal, sinkConfInit, toString
public DistCacheTap(cascading.tap.hadoop.Hfs parent)
parent
- an Hfs or GlobHfs instance representing a small file.protected Path[] getLocalCacheFiles(cascading.flow.FlowProcess<? extends Configuration> flowProcess) throws java.io.IOException
getLocalCacheFiles
in class cascading.tap.hadoop.BaseDistCacheTap
java.io.IOException
protected void addLocalCacheFiles(Configuration conf, java.net.URI uri)
addLocalCacheFiles
in class cascading.tap.hadoop.BaseDistCacheTap
Copyright © 2007-2015 Xplenty, Inc. All Rights Reserved.