cascading.flow.local
Class LocalFlowProcess

java.lang.Object
  extended by cascading.flow.FlowProcess<Properties>
      extended by cascading.flow.local.LocalFlowProcess

public class LocalFlowProcess
extends FlowProcess<Properties>

Class LocalFlowProcess is the local mode implementation of FlowProcess.


Nested Class Summary
 
Nested classes/interfaces inherited from class cascading.flow.FlowProcess
FlowProcess.NullFlowProcess
 
Field Summary
 
Fields inherited from class cascading.flow.FlowProcess
NULL
 
Constructor Summary
LocalFlowProcess()
           
LocalFlowProcess(FlowSession flowSession, Properties config)
           
LocalFlowProcess(LocalFlowProcess flowProcess, Properties properties)
           
LocalFlowProcess(Properties config)
           
 
Method Summary
 Properties copyConfig(Properties config)
           
 FlowProcess copyWith(Properties object)
           
 Map<String,String> diffConfigIntoMap(Properties defaultConfig, Properties updatedConfig)
           
 Properties getConfigCopy()
           
 int getCurrentSliceNum()
          Method getCurrentSliceNum returns an integer representing which slice instance currently running.
 int getNumProcessSlices()
          Method getNumProcessSlices returns the number of parallel slices or tasks allocated for this process execution.
 Object getProperty(String key)
          Method getProperty should be used to return configuration parameters from the underlying system.
 Collection<String> getPropertyKeys()
          Method getPropertyKeys returns an immutable collection of all available property key values.
 void increment(Enum counter, long amount)
          Method increment is used to increment a custom counter.
 void increment(String group, String counter, long amount)
          Method increment is used to increment a custom counter.
 boolean isCounterStatusInitialized()
          Method isCounterStatusInitialized returns true if it is safe to increment a counter or set a status.
 void keepAlive()
          Method keepAlive notifies the system that the current process is still alive.
 Properties mergeMapIntoConfig(Properties defaultConfig, Map<String,String> map)
           
 Object newInstance(String className)
          Method newInstance creates a new object instance from the given className argument delegating to any platform specific instantiation and configuration routines.
 TupleEntryCollector openSystemIntermediateForWrite()
           
 TupleEntryIterator openTapForRead(Tap tap)
          Method openTapForRead return a TupleEntryIterator for the given Tap instance.
 TupleEntryCollector openTapForWrite(Tap tap)
          Method openTapForWrite returns a (@link TupleCollector} for the given Tap instance.
 TupleEntryCollector openTrapForWrite(Tap trap)
          Method openTrapForWrite returns a (@link TupleCollector} for the given Tap instance.
 void setStatus(String status)
          Method setStatus is used to set the status of the current operation.
 void setStepStats(LocalStepStats stepStats)
           
 
Methods inherited from class cascading.flow.FlowProcess
getCurrentSession, getID, getIntegerProperty, getStringProperty, setCurrentSession
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

LocalFlowProcess

public LocalFlowProcess()

LocalFlowProcess

public LocalFlowProcess(Properties config)

LocalFlowProcess

public LocalFlowProcess(FlowSession flowSession,
                        Properties config)

LocalFlowProcess

public LocalFlowProcess(LocalFlowProcess flowProcess,
                        Properties properties)
Method Detail

setStepStats

public void setStepStats(LocalStepStats stepStats)

getNumProcessSlices

public int getNumProcessSlices()
Description copied from class: FlowProcess
Method getNumProcessSlices returns the number of parallel slices or tasks allocated for this process execution.

For MapReduce platforms, this is the same as the number of tasks for a given MapReduce job.

Specified by:
getNumProcessSlices in class FlowProcess<Properties>
Returns:
an int

getCurrentSliceNum

public int getCurrentSliceNum()
Description copied from class: FlowProcess
Method getCurrentSliceNum returns an integer representing which slice instance currently running.

0 (zero) is the first slice instance.

Specified by:
getCurrentSliceNum in class FlowProcess<Properties>
Returns:
an int

getProperty

public Object getProperty(String key)
Description copied from class: FlowProcess
Method getProperty should be used to return configuration parameters from the underlying system.

In the case of Hadoop, the current Configuration will be queried.

Specified by:
getProperty in class FlowProcess<Properties>
Parameters:
key - of type String
Returns:
an Object

getPropertyKeys

public Collection<String> getPropertyKeys()
Description copied from class: FlowProcess
Method getPropertyKeys returns an immutable collection of all available property key values.

Specified by:
getPropertyKeys in class FlowProcess<Properties>
Returns:
a Collection

newInstance

public Object newInstance(String className)
Description copied from class: FlowProcess
Method newInstance creates a new object instance from the given className argument delegating to any platform specific instantiation and configuration routines.

Specified by:
newInstance in class FlowProcess<Properties>
Returns:
an instance of className

keepAlive

public void keepAlive()
Description copied from class: FlowProcess
Method keepAlive notifies the system that the current process is still alive. Use this method if a particular Operation takes some moments to complete. Each system is different, so calling ping every few seconds to every minute or so would be best.

This method will fail silently if the underlying mechanism to notify keepAlive status are not initialized.

Specified by:
keepAlive in class FlowProcess<Properties>

increment

public void increment(Enum counter,
                      long amount)
Description copied from class: FlowProcess
Method increment is used to increment a custom counter. Counters must be of type Enum. The amount to increment must be a integer value.

This method will fail if the underlying counter infrastructure is unavailable. See FlowProcess.isCounterStatusInitialized().

Specified by:
increment in class FlowProcess<Properties>
Parameters:
counter - of type Enum
amount - of type int

increment

public void increment(String group,
                      String counter,
                      long amount)
Description copied from class: FlowProcess
Method increment is used to increment a custom counter. The amount to increment must be a integer value.

This method will fail if the underlying counter infrastructure is unavailable. See FlowProcess.isCounterStatusInitialized().

Specified by:
increment in class FlowProcess<Properties>
Parameters:
group - of type String
counter - of type String
amount - of type int

setStatus

public void setStatus(String status)
Description copied from class: FlowProcess
Method setStatus is used to set the status of the current operation.

This method will fail if the underlying counter infrastructure is unavailable. See FlowProcess.isCounterStatusInitialized().

Specified by:
setStatus in class FlowProcess<Properties>
Parameters:
status - of type String

isCounterStatusInitialized

public boolean isCounterStatusInitialized()
Description copied from class: FlowProcess
Method isCounterStatusInitialized returns true if it is safe to increment a counter or set a status.

Specified by:
isCounterStatusInitialized in class FlowProcess<Properties>
Returns:
boolean

openTapForRead

public TupleEntryIterator openTapForRead(Tap tap)
                                  throws IOException
Description copied from class: FlowProcess
Method openTapForRead return a TupleEntryIterator for the given Tap instance.

Note the returned iterator will return the same instance of TupleEntry on every call, thus a copy must be made of either the TupleEntry or the underlying Tuple instance if they are to be stored in a Collection.

Specified by:
openTapForRead in class FlowProcess<Properties>
Parameters:
tap - of type Tap
Returns:
TupleIterator
Throws:
IOException - when there is a failure opening the resource

openTapForWrite

public TupleEntryCollector openTapForWrite(Tap tap)
                                    throws IOException
Description copied from class: FlowProcess
Method openTapForWrite returns a (@link TupleCollector} for the given Tap instance.

Specified by:
openTapForWrite in class FlowProcess<Properties>
Parameters:
tap - of type Tap
Returns:
TupleCollector
Throws:
IOException - when there is a failure opening the resource

openTrapForWrite

public TupleEntryCollector openTrapForWrite(Tap trap)
                                     throws IOException
Description copied from class: FlowProcess
Method openTrapForWrite returns a (@link TupleCollector} for the given Tap instance.

Specified by:
openTrapForWrite in class FlowProcess<Properties>
Parameters:
trap - of type Tap
Returns:
TupleCollector
Throws:
IOException - when there is a failure opening the resource

openSystemIntermediateForWrite

public TupleEntryCollector openSystemIntermediateForWrite()
                                                   throws IOException
Specified by:
openSystemIntermediateForWrite in class FlowProcess<Properties>
Throws:
IOException

copyWith

public FlowProcess copyWith(Properties object)
Specified by:
copyWith in class FlowProcess<Properties>

getConfigCopy

public Properties getConfigCopy()
Specified by:
getConfigCopy in class FlowProcess<Properties>

copyConfig

public Properties copyConfig(Properties config)
Specified by:
copyConfig in class FlowProcess<Properties>

diffConfigIntoMap

public Map<String,String> diffConfigIntoMap(Properties defaultConfig,
                                            Properties updatedConfig)
Specified by:
diffConfigIntoMap in class FlowProcess<Properties>

mergeMapIntoConfig

public Properties mergeMapIntoConfig(Properties defaultConfig,
                                     Map<String,String> map)
Specified by:
mergeMapIntoConfig in class FlowProcess<Properties>


Copyright © 2007-2013 Concurrent, Inc. All Rights Reserved.