When creating custom Operations
(Function
,
Filter
,Aggregator
, or
Buffer
) do not store operation state in class
fields. For example, if implementing a custom 'counter'
Aggregator
, do not create a field named 'count'
and increment it on every
Aggregator.aggregate()
call. There is no
guarantee your Operation will be called from a single thread in a JVM,
future version of Hadoop could execute the same operation from multiple
threads.
To maintain state across Operation
calls,
create and initialize a "context" object that is maintained by the
appropriate OperationCall
(FilterCall
,FunctionCall
,
AggregatorCall
, and
BufferCall
). In the example above, store an
Integer 0 in the AggregatorCall
passed to the
Aggregator.start()
method and increment it in
the Aggregator.aggregate()
method.
Copyright © 2007-2008 Concurrent, Inc. All Rights Reserved.