11.10 Maintaining State in Operations

When creating custom Operations (Function, Filter, Aggregator, or Buffer) do not store operation state in class fields.

For example, if implementing a custom "counter" Aggregator, do not create a field named "count" and increment it on every Aggregator.aggregate() call. There is no guarantee that your Operation will be called from a single thread in a JVM - and future versions of Hadoop or Cascading local mode might execute the same operation from multiple threads.

Copyright © 2007-2012 Concurrent, Inc. All Rights Reserved.