9.4 Stream Ordering

Remove duplicate Tuples in a stream
// group on all values
pipe = new GroupBy( pipe, Fields.ALL );
// only take the first tuple in the grouping, ignore the rest
pipe = new Every( pipe, Fields.ALL, new First(), Fields.RESULTS );
Create a list of unique values
// find all unique 'ip' values
pipe = new Unique( pipe, new Fields( "ip" ) );
Find first occurrence in time of a unique value
// group on all unique 'ip' values
// secondary sort on 'datetime', natural order is in ascending order
pipe = new GroupBy( pipe, new Fields( "ip" ), new Fields( "datetime" ) );
// take the first 'ip' tuple in the group which has the
// oldest 'datetime' value
pipe = new Every( pipe, Fields.ALL, new First(), Fields.RESULTS );

Copyright © 2007-2008 Concurrent, Inc. All Rights Reserved.