Cascading 3.0 User Guide - Extending Cascading

Extending Cascading

Scripting

The Cascading API was designed with scripting in mind. Any JVM-compatible scripting language can import and instantiate Cascading classes, create pipe assemblies and Flows, and execute those Flows. And if the scripting language in question supports domain-specific language (DSL) creation, users can create their own DSLs to handle common idioms.

The Cascading website (http://cascading.org/extensions/) includes information on scripting language bindings that are publicly available.

Custom Types and Serialization

The Tuple class is a generic container for all java.lang.Object instances.

Thus any primitive value or custom class can be stored in a Tuple instance — that is, returned by a Function, Aggregator, or Buffer as a result value.

Unfortunately there is no common method for managing the serialization of custom types that is cross-platform. See the platform-specific topics of this User Guide documentation for details about registering serializers that Cascading can adopt at runtime.

Custom Comparators and Hashing

Frequently, objects in one Tuple are compared to objects in a second Tuple. This is especially true during the sort phase of GroupBy and CoGroup. By default, Cascading uses the equals() and hashCode() Object native methods to compare two values and get a consistent hash code for a given value, respectively.

There are two different approaches that you can take to override the default behavior:

Create a java.util.Comparator class to perform comparisons on given field in a Tuple. For instance, to secondary-sort a collection of custom Person objects in a GroupBy, use the Fields.setComparator() method to designate the custom Comparator to the Fields instance that specifies the sort fields.
Alternatively, you can set a default Comparator for a Flow or for a local Pipe instance by one of the following ways:
- Either calling FlowProps.setDefaultTupleElementComparator() on a Properties instance
- Or using the cascading.flow.tuple.element.comparator property key

If the hash code must also be customized, the custom Comparator can implement the cascading.tuple.Hasher interface.

For more information, see the Javadoc.