cascading.pipe.assembly

Class MaxBy

java.lang.Object

cascading.pipe.Pipe

cascading.pipe.SubAssembly

cascading.pipe.assembly.AggregateBy

cascading.pipe.assembly.MaxBy

All Implemented Interfaces:

FlowElement, Traceable, Serializable

public class MaxBy
extends AggregateBy

Class MaxBy is used to find the maximum value in a grouping.

Typically finding the max value of a field in a tuple stream relies on a GroupBy and a MaxValue Aggregator operation.

This SubAssembly also uses the MaxBy.MaxPartials AggregateBy.Functor to track the maximum value before the GroupBy operator to reduce IO over the network.

This strategy is similar to using combiners, except no sorting or serialization is invoked and results in a much simpler mechanism.

The threshold value tells the underlying MaxPartials functions how many unique key sums to accumulate in the LRU cache, before emitting the least recently used entry. This accumulation happens map-side, and thus is bounded by the size of your map task JVM and the typical size of each group key.

By default, either the value of AggregateByProps.AGGREGATE_BY_CAPACITY System property or AggregateByProps.AGGREGATE_BY_DEFAULT_CAPACITY will be used.

See Also:

AggregateBy, Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MaxBy.MaxPartials

Nested classes/interfaces inherited from class cascading.pipe.assembly.AggregateBy

AggregateBy.Cache, AggregateBy.CompositeFunction, AggregateBy.Functor

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_THRESHOLD Deprecated.

Fields inherited from class cascading.pipe.assembly.AggregateBy

USE_DEFAULT_THRESHOLD

Fields inherited from class cascading.pipe.Pipe

configDef, nodeConfigDef, parent, stepConfigDef

Constructor Summary

Constructors

Constructor and Description
MaxBy(Fields valueField, Fields maxField) Constructor MaxBy creates a new MaxBy instance.
MaxBy(Pipe[] pipes, Fields groupingFields, Fields valueField, Fields maxField) Constructor MaxBy creates a new MaxBy instance.
MaxBy(Pipe[] pipes, Fields groupingFields, Fields valueField, Fields maxField, int threshold) Constructor MaxBy creates a new MaxBy instance.
MaxBy(Pipe pipe, Fields groupingFields, Fields valueField, Fields maxField) Constructor MaxBy creates a new MaxBy instance.
MaxBy(Pipe pipe, Fields groupingFields, Fields valueField, Fields maxField, int threshold) Constructor MaxBy creates a new MaxBy instance.
MaxBy(String name, Pipe[] pipes, Fields groupingFields, Fields valueField, Fields maxField) Constructor MaxBy creates a new MaxBy instance.
MaxBy(String name, Pipe[] pipes, Fields groupingFields, Fields valueField, Fields maxField, int threshold) Constructor MaxBy creates a new MaxBy instance.
MaxBy(String name, Pipe pipe, Fields groupingFields, Fields valueField, Fields maxField) Constructor MaxBy creates a new MaxBy instance.
MaxBy(String name, Pipe pipe, Fields groupingFields, Fields valueField, Fields maxField, int threshold) Constructor MaxBy creates a new MaxBy instance.

Method Summary

Methods inherited from class cascading.pipe.assembly.AggregateBy

getAggregators, getArgumentFields, getCapacity, getFieldDeclarations, getFunctors, getGroupBy, getGroupingFields, initialize, initialize, verify

Methods inherited from class cascading.pipe.SubAssembly

getName, getPrevious, getTailNames, getTails, setPrevious, setTails, unwind

Methods inherited from class cascading.pipe.Pipe

equals, getConfigDef, getHeads, getNodeConfigDef, getParent, getStepConfigDef, getTrace, hasConfigDef, hashCode, hasNodeConfigDef, hasStepConfigDef, id, isEquivalentTo, named, names, outgoingScopeFor, pipes, print, printInternal, resolveIncomingOperationArgumentFields, resolveIncomingOperationPassThroughFields, setParent, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

DEFAULT_THRESHOLD

@Deprecated
public static final int DEFAULT_THRESHOLD

Deprecated.

DEFAULT_THRESHOLD

See Also:

Constant Field Values

Constructor Detail

MaxBy

@ConstructorProperties(value={"valueField","maxField"})
public MaxBy(Fields valueField,
                                Fields maxField)

Constructor MaxBy creates a new MaxBy instance. Use this constructor when used with a AggregateBy instance.

Parameters:

valueField - of type Fields

maxField - of type Fields

MaxBy

@ConstructorProperties(value={"pipe","groupingFields","valueField","maxField"})
public MaxBy(Pipe pipe,
                                Fields groupingFields,
                                Fields valueField,
                                Fields maxField)

Constructor MaxBy creates a new MaxBy instance.

Parameters:

pipe - of type Pipe

groupingFields - of type Fields

valueField - of type Fields

maxField - of type Fields

MaxBy

@ConstructorProperties(value={"pipe","groupingFields","valueField","maxField","threshold"})
public MaxBy(Pipe pipe,
                                Fields groupingFields,
                                Fields valueField,
                                Fields maxField,
                                int threshold)

Constructor MaxBy creates a new MaxBy instance.

Parameters:

pipe - of type Pipe

groupingFields - of type Fields

valueField - of type Fields

maxField - of type Fields

threshold - of type int

MaxBy

@ConstructorProperties(value={"name","pipe","groupingFields","valueField","maxField"})
public MaxBy(String name,
                                Pipe pipe,
                                Fields groupingFields,
                                Fields valueField,
                                Fields maxField)

Constructor MaxBy creates a new MaxBy instance.

Parameters:

name - of type String

pipe - of type Pipe

groupingFields - of type Fields

valueField - of type Fields

maxField - of type Fields

MaxBy

@ConstructorProperties(value={"name","pipe","groupingFields","valueField","maxField","threshold"})
public MaxBy(String name,
                                Pipe pipe,
                                Fields groupingFields,
                                Fields valueField,
                                Fields maxField,
                                int threshold)

Constructor MaxBy creates a new MaxBy instance.

Parameters:

name - of type String

pipe - of type Pipe

groupingFields - of type Fields

valueField - of type Fields

maxField - of type Fields

threshold - of type int

MaxBy

@ConstructorProperties(value={"pipes","groupingFields","valueField","maxField"})
public MaxBy(Pipe[] pipes,
                                Fields groupingFields,
                                Fields valueField,
                                Fields maxField)

Constructor MaxBy creates a new MaxBy instance.

Parameters:

pipes - of type Pipe[]

groupingFields - of type Fields

valueField - of type Fields

maxField - of type Fields

MaxBy

@ConstructorProperties(value={"pipes","groupingFields","valueField","maxField","threshold"})
public MaxBy(Pipe[] pipes,
                                Fields groupingFields,
                                Fields valueField,
                                Fields maxField,
                                int threshold)

Constructor MaxBy creates a new MaxBy instance.

Parameters:

pipes - of type Pipe[]

groupingFields - of type Fields

valueField - of type Fields

maxField - of type Fields

threshold - of type int

MaxBy

@ConstructorProperties(value={"name","pipes","groupingFields","valueField","maxField"})
public MaxBy(String name,
                                Pipe[] pipes,
                                Fields groupingFields,
                                Fields valueField,
                                Fields maxField)

Constructor MaxBy creates a new MaxBy instance.

Parameters:

name - of type String

pipes - of type Pipe[]

groupingFields - of type Fields

valueField - of type Fields

maxField - of type Fields

MaxBy

@ConstructorProperties(value={"name","pipes","groupingFields","valueField","maxField","threshold"})
public MaxBy(String name,
                                Pipe[] pipes,
                                Fields groupingFields,
                                Fields valueField,
                                Fields maxField,
                                int threshold)

Constructor MaxBy creates a new MaxBy instance.

Parameters:

name - of type String

pipes - of type Pipe[]

groupingFields - of type Fields

valueField - of type Fields

maxField - of type Fields

threshold - of type int