public class FirstBy extends AggregateBy
GroupBy
and a
First
Aggregator
operation.
If the firstFields
argument has custom Comparator
instances, they will be used
as the GroupBy sortFields
.
This SubAssembly also uses the FirstBy.FirstPartials
AggregateBy.Functor
to collect field values before the GroupBy operator to reduce IO over the network.
This strategy is similar to using combiners
, except no sorting or serialization is invoked and results
in a much simpler mechanism.
The threshold
value tells the underlying FirstPartials functions how many unique key counts to accumulate
in the LRU cache, before emitting the least recently used entry. This accumulation happens map-side, and thus is
bounded by the size of your map task JVM and the typical size of each group key.
By default, either the value of AggregateByProps.AGGREGATE_BY_CAPACITY
System property
or AggregateByProps.AGGREGATE_BY_DEFAULT_CAPACITY
will be used.AggregateBy
,
Serialized FormModifier and Type | Class and Description |
---|---|
static class |
FirstBy.FirstPartials
Class CountPartials is a
AggregateBy.Functor that is used to count observed duplicates from the tuple stream. |
AggregateBy.Cache, AggregateBy.CompositeFunction, AggregateBy.Functor
USE_DEFAULT_THRESHOLD
configDef, nodeConfigDef, parent, stepConfigDef
Constructor and Description |
---|
FirstBy(Fields firstFields)
Constructor FirstBy creates a new FirstBy instance.
|
FirstBy(Fields argumentFields,
Fields firstFields)
Constructor FirstBy creates a new FirstBy instance.
|
FirstBy(Pipe[] pipes,
Fields groupingFields,
Fields firstFields)
Constructor FirstBy creates a new FirstBy instance.
|
FirstBy(Pipe[] pipes,
Fields groupingFields,
Fields firstFields,
int threshold)
Constructor FirstBy creates a new FirstBy instance.
|
FirstBy(Pipe pipe,
Fields groupingFields,
Fields firstFields)
Constructor FirstBy creates a new FirstBy instance.
|
FirstBy(Pipe pipe,
Fields groupingFields,
Fields firstFields,
int threshold)
Constructor FirstBy creates a new FirstBy instance.
|
FirstBy(java.lang.String name,
Pipe[] pipes,
Fields groupingFields,
Fields firstFields)
Constructor FirstBy creates a new FirstBy instance.
|
FirstBy(java.lang.String name,
Pipe[] pipes,
Fields groupingFields,
Fields firstFields,
int threshold)
Constructor FirstBy creates a new FirstBy instance.
|
FirstBy(java.lang.String name,
Pipe pipe,
Fields groupingFields,
Fields firstFields)
Constructor FirstBy creates a new FirstBy instance.
|
FirstBy(java.lang.String name,
Pipe pipe,
Fields groupingFields,
Fields firstFields,
int threshold)
Constructor FirstBy creates a new FirstBy instance.
|
getAggregators, getArgumentFields, getCapacity, getFieldDeclarations, getFunctors, getGroupBy, getGroupingFields, initialize, initialize, verify
getName, getPrevious, getTailNames, getTails, setPrevious, setTails, unwind
equals, getConfigDef, getHeads, getNodeConfigDef, getParent, getStepConfigDef, getTrace, hasConfigDef, hashCode, hasNodeConfigDef, hasStepConfigDef, id, named, names, outgoingScopeFor, pipes, print, printInternal, resolveIncomingOperationArgumentFields, resolveIncomingOperationPassThroughFields, setParent, toString
@ConstructorProperties(value="firstFields") public FirstBy(Fields firstFields)
AggregateBy
instance.firstFields
- of type Fields@ConstructorProperties(value={"argumentFields","firstFields"}) public FirstBy(Fields argumentFields, Fields firstFields)
AggregateBy
instance.firstFields
- of type Fields@ConstructorProperties(value={"pipe","groupingFields","firstFields"}) public FirstBy(Pipe pipe, Fields groupingFields, Fields firstFields)
pipe
- of type PipegroupingFields
- of type FieldsfirstFields
- of type Fields@ConstructorProperties(value={"pipe","groupingFields","firstFields","threshold"}) public FirstBy(Pipe pipe, Fields groupingFields, Fields firstFields, int threshold)
pipe
- of type PipegroupingFields
- of type FieldsfirstFields
- fo type Fieldsthreshold
- of type int@ConstructorProperties(value={"name","pipe","groupingFields","firstFields"}) public FirstBy(java.lang.String name, Pipe pipe, Fields groupingFields, Fields firstFields)
name
- of type Stringpipe
- of type PipegroupingFields
- of type FieldsfirstFields
- of type Fields@ConstructorProperties(value={"name","pipe","groupingFields","firstFields","threshold"}) public FirstBy(java.lang.String name, Pipe pipe, Fields groupingFields, Fields firstFields, int threshold)
name
- of type Stringpipe
- of type PipegroupingFields
- of type FieldsfirstFields
- of type Fieldsthreshold
- of type int@ConstructorProperties(value={"pipes","groupingFields","firstFields"}) public FirstBy(Pipe[] pipes, Fields groupingFields, Fields firstFields)
pipes
- of type Pipe[]groupingFields
- of type FieldsfirstFields
- of type Fields@ConstructorProperties(value={"pipes","groupingFields","firstFields","threshold"}) public FirstBy(Pipe[] pipes, Fields groupingFields, Fields firstFields, int threshold)
pipes
- of type Pipe[]groupingFields
- of type FieldsfirstFields
- of type Fieldsthreshold
- of type int@ConstructorProperties(value={"name","pipes","groupingFields","firstFields"}) public FirstBy(java.lang.String name, Pipe[] pipes, Fields groupingFields, Fields firstFields)
name
- of type Stringpipes
- of type Pipe[]groupingFields
- of type FieldsfirstFields
- of type Fields@ConstructorProperties(value={"name","pipes","groupingFields","firstFields","threshold"}) public FirstBy(java.lang.String name, Pipe[] pipes, Fields groupingFields, Fields firstFields, int threshold)
name
- of type Stringpipes
- of type Pipe[]groupingFields
- of type FieldsfirstFields
- of type Fieldsthreshold
- of type intCopyright © 2007-2015 Xplenty, Inc. All Rights Reserved.