|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object cascading.tuple.collect.SpillableTupleList cascading.tuple.hadoop.collect.HadoopSpillableTupleList
public class HadoopSpillableTupleList
SpillableTupleList is a simple Iterable
object that can store an unlimited number of Tuple
instances by spilling
excess to a temporary disk file.
defaultCodecs
values. To disable compression or
change the codecs, see SpillableProps.SPILL_COMPRESS
and SpillableProps.SPILL_CODECS
.
It is recommended to add Lzo if available.
"org.apache.hadoop.io.compress.LzoCodec,org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec"
Nested Class Summary |
---|
Nested classes/interfaces inherited from interface cascading.tuple.collect.Spillable |
---|
cascading.tuple.collect.Spillable.SpillListener, cascading.tuple.collect.Spillable.SpillStrategy |
Field Summary | |
---|---|
static String |
defaultCodecs
|
Fields inherited from class cascading.tuple.collect.SpillableTupleList |
---|
SPILL_CODECS, SPILL_COMPRESS, SPILL_THRESHOLD |
Constructor Summary | |
---|---|
HadoopSpillableTupleList(int threshold,
CompressionCodec codec,
JobConf jobConf)
Constructor SpillableTupleList creates a new SpillableTupleList instance using the given threshold value, and the first available compression codec, if any. |
|
HadoopSpillableTupleList(int threshold,
TupleSerialization tupleSerialization,
CompressionCodec codec)
|
Method Summary | |
---|---|
protected cascading.tuple.io.TupleInputStream |
createTupleInputStream(File file)
|
protected cascading.tuple.io.TupleOutputStream |
createTupleOutputStream(File file)
|
static CompressionCodec |
getCodec(cascading.flow.FlowProcess flowProcess,
String defaultCodecs)
|
Methods inherited from class cascading.tuple.collect.SpillableTupleList |
---|
add, addAll, clear, contains, containsAll, getCodecClass, getGrouping, getThreshold, isEmpty, iterator, remove, removeAll, retainAll, setGrouping, setSpillListener, setSpillStrategy, size, spillCount, toArray, toArray |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Methods inherited from interface java.util.Collection |
---|
equals, hashCode |
Field Detail |
---|
public static final String defaultCodecs
Constructor Detail |
---|
public HadoopSpillableTupleList(int threshold, CompressionCodec codec, JobConf jobConf)
threshold
- of type longcodec
- of type CompressionCodecpublic HadoopSpillableTupleList(int threshold, TupleSerialization tupleSerialization, CompressionCodec codec)
Method Detail |
---|
public static CompressionCodec getCodec(cascading.flow.FlowProcess flowProcess, String defaultCodecs)
protected cascading.tuple.io.TupleOutputStream createTupleOutputStream(File file)
createTupleOutputStream
in class cascading.tuple.collect.SpillableTupleList
protected cascading.tuple.io.TupleInputStream createTupleInputStream(File file)
createTupleInputStream
in class cascading.tuple.collect.SpillableTupleList
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |