cascading.pipe.assembly
Class SumBy

java.lang.Object
  extended by cascading.pipe.Pipe
      extended by cascading.pipe.SubAssembly
          extended by cascading.pipe.assembly.AggregateBy
              extended by cascading.pipe.assembly.SumBy
All Implemented Interfaces:
FlowElement, Serializable

public class SumBy
extends AggregateBy

Class SumBy is used to sum values associated with duplicate keys in a tuple stream.

Typically finding the sum of field in a tuple stream relies on a GroupBy and a Sum Aggregator operation.

If all the values to be summed are all null, the result value is a function of how null is coerced by the given sumType. If a primitive type, 0 will be returned. Otherwise null.

This SubAssembly also uses the SumBy.SumPartials AggregateBy.Functor to sum field values before the GroupBy operator to reduce IO over the network.

This strategy is similar to using combiners, except no sorting or serialization is invoked and results in a much simpler mechanism.

The threshold value tells the underlying SumPartials functions how many unique key sums to accumulate in the LRU cache, before emitting the least recently used entry.

By default, either the value of AggregateBy.AGGREGATE_BY_THRESHOLD System property or AggregateBy.DEFAULT_THRESHOLD will be used.

See Also:
AggregateBy, Serialized Form

Nested Class Summary
static class SumBy.SumPartials
          Class SumPartials is a AggregateBy.Functor that is used to sum observed duplicates from the tuple stream.
 
Nested classes/interfaces inherited from class cascading.pipe.assembly.AggregateBy
AggregateBy.CompositeFunction, AggregateBy.Flush, AggregateBy.Functor
 
Field Summary
static int DEFAULT_THRESHOLD
          Deprecated. 
 
Fields inherited from class cascading.pipe.assembly.AggregateBy
AGGREGATE_BY_THRESHOLD, USE_DEFAULT_THRESHOLD
 
Fields inherited from class cascading.pipe.Pipe
configDef, parent, stepConfigDef
 
Constructor Summary
SumBy(Fields valueField, Fields sumField)
          Constructor SumBy creates a new SumBy instance.
SumBy(Fields valueField, Fields sumField, Class sumType)
          Constructor SumBy creates a new SumBy instance.
SumBy(Pipe[] pipes, Fields groupingFields, Fields valueField, Fields sumField)
          Constructor SumBy creates a new SumBy instance.
SumBy(Pipe[] pipes, Fields groupingFields, Fields valueField, Fields sumField, Class sumType)
          Constructor SumBy creates a new SumBy instance.
SumBy(Pipe[] pipes, Fields groupingFields, Fields valueField, Fields sumField, Class sumType, int threshold)
          Constructor SumBy creates a new SumBy instance.
SumBy(Pipe[] pipes, Fields groupingFields, Fields valueField, Fields sumField, int threshold)
          Constructor SumBy creates a new SumBy instance.
SumBy(Pipe pipe, Fields groupingFields, Fields valueField, Fields sumField)
          Constructor SumBy creates a new SumBy instance.
SumBy(Pipe pipe, Fields groupingFields, Fields valueField, Fields sumField, Class sumType)
          Constructor SumBy creates a new SumBy instance.
SumBy(Pipe pipe, Fields groupingFields, Fields valueField, Fields sumField, Class sumType, int threshold)
          Constructor SumBy creates a new SumBy instance.
SumBy(Pipe pipe, Fields groupingFields, Fields valueField, Fields sumField, int threshold)
          Constructor SumBy creates a new SumBy instance.
SumBy(String name, Pipe[] pipes, Fields groupingFields, Fields valueField, Fields sumField)
          Constructor SumBy creates a new SumBy instance.
SumBy(String name, Pipe[] pipes, Fields groupingFields, Fields valueField, Fields sumField, Class sumType)
          Constructor SumBy creates a new SumBy instance.
SumBy(String name, Pipe[] pipes, Fields groupingFields, Fields valueField, Fields sumField, Class sumType, int threshold)
          Constructor SumBy creates a new SumBy instance.
SumBy(String name, Pipe[] pipes, Fields groupingFields, Fields valueField, Fields sumField, int threshold)
          Constructor SumBy creates a new SumBy instance.
SumBy(String name, Pipe pipe, Fields groupingFields, Fields valueField, Fields sumField)
          Constructor SumBy creates a new SumBy instance.
SumBy(String name, Pipe pipe, Fields groupingFields, Fields valueField, Fields sumField, Class sumType)
          Constructor SumBy creates a new SumBy instance.
SumBy(String name, Pipe pipe, Fields groupingFields, Fields valueField, Fields sumField, Class sumType, int threshold)
          Constructor SumBy creates a new SumBy instance.
SumBy(String name, Pipe pipe, Fields groupingFields, Fields valueField, Fields sumField, int threshold)
          Constructor SumBy creates a new SumBy instance.
 
Method Summary
 
Methods inherited from class cascading.pipe.assembly.AggregateBy
getAggregators, getArgumentFields, getFieldDeclarations, getFunctors, getGroupBy, getGroupingFields, initialize, initialize, verify
 
Methods inherited from class cascading.pipe.SubAssembly
getName, getPrevious, getTailNames, getTails, setPrevious, setTails, unwind
 
Methods inherited from class cascading.pipe.Pipe
equals, getConfigDef, getHeads, getParent, getStepConfigDef, getTrace, hasConfigDef, hashCode, hasStepConfigDef, id, isEquivalentTo, named, names, outgoingScopeFor, pipes, print, printInternal, resolveIncomingOperationArgumentFields, resolveIncomingOperationPassThroughFields, setParent, toString
 
Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait
 

Field Detail

DEFAULT_THRESHOLD

@Deprecated
public static final int DEFAULT_THRESHOLD
Deprecated. 
DEFAULT_THRESHOLD

See Also:
Constant Field Values
Constructor Detail

SumBy

@ConstructorProperties(value={"valueField","sumField"})
public SumBy(Fields valueField,
                                        Fields sumField)
Constructor SumBy creates a new SumBy instance. Use this constructor when used with a AggregateBy instance.

Parameters:
valueField - of type Fields
sumField - of type Fields

SumBy

@ConstructorProperties(value={"pipe","groupingFields","valueField","sumField"})
public SumBy(Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField)
Constructor SumBy creates a new SumBy instance.

Parameters:
pipe - of type Pipe
groupingFields - of type Fields
valueField - of type Fields
sumField - of type Fields

SumBy

@ConstructorProperties(value={"pipe","groupingFields","valueField","sumField","threshold"})
public SumBy(Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        int threshold)
Constructor SumBy creates a new SumBy instance.

Parameters:
pipe - of type Pipe
groupingFields - of type Fields
valueField - of type Fields
sumField - of type Fields
threshold - of type int

SumBy

@ConstructorProperties(value={"name","pipe","groupingFields","valueField","sumField"})
public SumBy(String name,
                                        Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField)
Constructor SumBy creates a new SumBy instance.

Parameters:
name - of type String
pipe - of type Pipe
groupingFields - of type Fields
valueField - of type Fields
sumField - of type Fields

SumBy

@ConstructorProperties(value={"name","pipe","groupingFields","valueField","sumField","threshold"})
public SumBy(String name,
                                        Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        int threshold)
Constructor SumBy creates a new SumBy instance.

Parameters:
name - of type String
pipe - of type Pipe
groupingFields - of type Fields
valueField - of type Fields
sumField - of type Fields
threshold - of type int

SumBy

@ConstructorProperties(value={"name","pipes","groupingFields","valueField","sumField"})
public SumBy(Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField)
Constructor SumBy creates a new SumBy instance.

Parameters:
pipes - of type Pipe[]
groupingFields - of type Fields
valueField - of type Fields
sumField - of type Fields

SumBy

@ConstructorProperties(value={"name","pipes","groupingFields","valueField","sumField","threshold"})
public SumBy(Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        int threshold)
Constructor SumBy creates a new SumBy instance.

Parameters:
pipes - of type Pipe[]
groupingFields - of type Fields
valueField - of type Fields
sumField - of type Fields
threshold - of type int

SumBy

@ConstructorProperties(value={"name","pipes","groupingFields","valueField","sumField"})
public SumBy(String name,
                                        Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField)
Constructor SumBy creates a new SumBy instance.

Parameters:
name - of type String
pipes - of type Pipe[]
groupingFields - of type Fields
valueField - of type Fields
sumField - of type Fields

SumBy

@ConstructorProperties(value={"name","pipes","groupingFields","valueField","sumField","threshold"})
public SumBy(String name,
                                        Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        int threshold)
Constructor SumBy creates a new SumBy instance.

Parameters:
name - of type String
pipes - of type Pipe[]
groupingFields - of type Fields
valueField - of type Fields
sumField - of type Fields
threshold - of type int

SumBy

@ConstructorProperties(value={"valueField","sumField","sumType"})
public SumBy(Fields valueField,
                                        Fields sumField,
                                        Class sumType)
Constructor SumBy creates a new SumBy instance. Use this constructor when used with a AggregateBy instance.

Parameters:
valueField - of type Fields
sumField - of type Fields
sumType - of type Class

SumBy

@ConstructorProperties(value={"pipe","groupingFields","valueField","sumField","sumType"})
public SumBy(Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        Class sumType)
Constructor SumBy creates a new SumBy instance.

Parameters:
pipe - of type Pipe
groupingFields - of type Fields
valueField - of type Fields
sumField - of type Fields
sumType - of type Class

SumBy

@ConstructorProperties(value={"pipe","groupingFields","valueField","sumField","sumType","threshold"})
public SumBy(Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        Class sumType,
                                        int threshold)
Constructor SumBy creates a new SumBy instance.

Parameters:
pipe - of type Pipe
groupingFields - of type Fields
valueField - of type Fields
sumField - of type Fields
sumType - of type Class
threshold - of type int

SumBy

@ConstructorProperties(value={"name","pipe","groupingFields","valueField","sumField","sumType"})
public SumBy(String name,
                                        Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        Class sumType)
Constructor SumBy creates a new SumBy instance.

Parameters:
name - of type String
pipe - of type Pipe
groupingFields - of type Fields
valueField - of type Fields
sumField - of type Fields
sumType - of type Class

SumBy

@ConstructorProperties(value={"name","pipe","groupingFields","valueField","sumField","sumType","threshold"})
public SumBy(String name,
                                        Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        Class sumType,
                                        int threshold)
Constructor SumBy creates a new SumBy instance.

Parameters:
name - of type String
pipe - of type Pipe
groupingFields - of type Fields
valueField - of type Fields
sumField - of type Fields
sumType - of type Class
threshold - of type int

SumBy

@ConstructorProperties(value={"name","pipes","groupingFields","valueField","sumField","sumType"})
public SumBy(Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        Class sumType)
Constructor SumBy creates a new SumBy instance.

Parameters:
pipes - of type Pipe[]
groupingFields - of type Fields
valueField - of type Fields
sumField - of type Fields
sumType - of type Class

SumBy

@ConstructorProperties(value={"name","pipes","groupingFields","valueField","sumField","sumType","threshold"})
public SumBy(Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        Class sumType,
                                        int threshold)
Constructor SumBy creates a new SumBy instance.

Parameters:
pipes - of type Pipe[]
groupingFields - of type Fields
valueField - of type Fields
sumField - of type Fields
sumType - of type Class
threshold - of type int

SumBy

@ConstructorProperties(value={"name","pipes","groupingFields","valueField","sumField","sumType"})
public SumBy(String name,
                                        Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        Class sumType)
Constructor SumBy creates a new SumBy instance.

Parameters:
name - of type String
pipes - of type Pipe[]
groupingFields - of type Fields
valueField - of type Fields
sumField - of type Fields
sumType - of type Class

SumBy

@ConstructorProperties(value={"name","pipes","groupingFields","valueField","sumField","sumType","threshold"})
public SumBy(String name,
                                        Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        Class sumType,
                                        int threshold)
Constructor SumBy creates a new SumBy instance.

Parameters:
name - of type String
pipes - of type Pipe[]
groupingFields - of type Fields
valueField - of type Fields
sumField - of type Fields
sumType - of type Class
threshold - of type int


Copyright © 2007-2013 Concurrent, Inc. All Rights Reserved.