|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object cascading.pipe.Pipe cascading.pipe.SubAssembly cascading.pipe.assembly.AggregateBy cascading.pipe.assembly.CountBy
public class CountBy
Class CountBy is used to count duplicates in a tuple stream.
Typically finding the count of a field in a tuple stream relies on aGroupBy
and a Count
Aggregator
operation.
This SubAssembly also uses the CountBy.CountPartials
AggregateBy.Functor
to count as many observed duplicates before the GroupBy operator to reduce IO over the network.
This strategy is similar to using combiners
, except no sorting or serialization is invoked and results
in a much simpler mechanism.
The threshold
value tells the underlying CountPartials functions how many values to cache for each
unique key before dropping values from the LRU cache.
AggregateBy
,
Serialized FormNested Class Summary | |
---|---|
static class |
CountBy.CountPartials
Class CountPartials is a AggregateBy.Functor that is used to count observed duplicates from the tuple stream. |
Nested classes/interfaces inherited from class cascading.pipe.assembly.AggregateBy |
---|
AggregateBy.CompositeFunction, AggregateBy.Functor |
Field Summary |
---|
Fields inherited from class cascading.pipe.Pipe |
---|
previous |
Constructor Summary | |
---|---|
CountBy(Fields countField)
Constructor CountBy creates a new CountBy instance. |
|
CountBy(Pipe[] pipes,
Fields groupingFields,
Fields countField)
Constructor CountBy creates a new CountBy instance. |
|
CountBy(Pipe[] pipes,
Fields groupingFields,
Fields countField,
int threshold)
Constructor CountBy creates a new CountBy instance. |
|
CountBy(Pipe pipe,
Fields groupingFields,
Fields countField)
Constructor CountBy creates a new CountBy instance. |
|
CountBy(Pipe pipe,
Fields groupingFields,
Fields countField,
int threshold)
Constructor CountBy creates a new CountBy instance. |
|
CountBy(String name,
Pipe[] pipes,
Fields groupingFields,
Fields countField)
Constructor CountBy creates a new CountBy instance. |
|
CountBy(String name,
Pipe[] pipes,
Fields groupingFields,
Fields countField,
int threshold)
Constructor CountBy creates a new CountBy instance. |
|
CountBy(String name,
Pipe pipe,
Fields groupingFields,
Fields countField)
Constructor CountBy creates a new CountBy instance. |
|
CountBy(String name,
Pipe pipe,
Fields groupingFields,
Fields countField,
int threshold)
Constructor CountBy creates a new CountBy instance. |
Method Summary |
---|
Methods inherited from class cascading.pipe.assembly.AggregateBy |
---|
getAggregators, getArgumentFields, getFunctors, initialize, initialize, verify |
Methods inherited from class cascading.pipe.SubAssembly |
---|
getName, getPrevious, getTailNames, getTails, setTails, unwind |
Methods inherited from class cascading.pipe.Pipe |
---|
equals, getHeads, getTrace, hashCode, isEquivalentTo, names, outgoingScopeFor, pipes, print, printInternal, resolveFields, resolveIncomingOperationFields, toString |
Methods inherited from class java.lang.Object |
---|
clone, finalize, getClass, notify, notifyAll, wait, wait, wait |
Constructor Detail |
---|
@ConstructorProperties(value="countField") public CountBy(Fields countField)
AggregateBy
instance.
countField
- of type Fields@ConstructorProperties(value={"pipe","groupingFields","countField"}) public CountBy(Pipe pipe, Fields groupingFields, Fields countField)
pipe
- of type PipegroupingFields
- of type FieldscountField
- of type Fields@ConstructorProperties(value={"pipe","groupingFields","countField","threshold"}) public CountBy(Pipe pipe, Fields groupingFields, Fields countField, int threshold)
pipe
- of type PipegroupingFields
- of type FieldscountField
- fo type Fieldsthreshold
- of type int@ConstructorProperties(value={"name","pipe","groupingFields","countField"}) public CountBy(String name, Pipe pipe, Fields groupingFields, Fields countField)
name
- of type Stringpipe
- of type PipegroupingFields
- of type FieldscountField
- of type Fields@ConstructorProperties(value={"name","pipe","groupingFields","countField","threshold"}) public CountBy(String name, Pipe pipe, Fields groupingFields, Fields countField, int threshold)
name
- of type Stringpipe
- of type PipegroupingFields
- of type FieldscountField
- of type Fieldsthreshold
- of type int@ConstructorProperties(value={"pipes","groupingFields","countField"}) public CountBy(Pipe[] pipes, Fields groupingFields, Fields countField)
pipes
- of type Pipe[]groupingFields
- of type FieldscountField
- of type Fields@ConstructorProperties(value={"pipes","groupingFields","countField","threshold"}) public CountBy(Pipe[] pipes, Fields groupingFields, Fields countField, int threshold)
pipes
- of type Pipe[]groupingFields
- of type FieldscountField
- of type Fieldsthreshold
- of type int@ConstructorProperties(value={"name","pipes","groupingFields","countField"}) public CountBy(String name, Pipe[] pipes, Fields groupingFields, Fields countField)
name
- of type Stringpipes
- of type Pipe[]groupingFields
- of type FieldscountField
- of type Fields@ConstructorProperties(value={"name","pipes","groupingFields","countField","threshold"}) public CountBy(String name, Pipe[] pipes, Fields groupingFields, Fields countField, int threshold)
name
- of type Stringpipes
- of type Pipe[]groupingFields
- of type FieldscountField
- of type Fieldsthreshold
- of type int
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |