cascading.flow.local
Class LocalFlowProcess

java.lang.Object
  cascading.flow.FlowProcess<Properties>
      cascading.flow.local.LocalFlowProcess

public class LocalFlowProcess
extends FlowProcess<Properties>

Class LocalFlowProcess is the local mode implementation of FlowProcess.

Nested Class Summary

Nested classes/interfaces inherited from class cascading.flow.FlowProcess

FlowProcess.NullFlowProcess

Field Summary

Fields inherited from class cascading.flow.FlowProcess

NULL

Constructor Summary

LocalFlowProcess()

LocalFlowProcess(FlowSession flowSession, Properties config)

LocalFlowProcess(LocalFlowProcess flowProcess, Properties properties)

LocalFlowProcess(Properties config)

Method Summary

Properties	copyConfig(Properties config)
FlowProcess	copyWith(Properties object)
Map<String,String>	diffConfigIntoMap(Properties defaultConfig, Properties updatedConfig)
Properties	getConfigCopy()
int	getCurrentSliceNum() Method getCurrentSliceNum returns an integer representing which slice instance currently running.
int	getNumProcessSlices() Method getNumProcessSlices returns the number of parallel slices or tasks allocated for this process execution.
Object	getProperty(String key) Method getProperty should be used to return configuration parameters from the underlying system.
Collection<String>	getPropertyKeys() Method getPropertyKeys returns an immutable collection of all available property key values.
void	increment(Enum counter, long amount) Method increment is used to increment a custom counter.
void	increment(String group, String counter, long amount) Method increment is used to increment a custom counter.
boolean	isCounterStatusInitialized() Method isCounterStatusInitialized returns true if it is safe to increment a counter or set a status.
void	keepAlive() Method keepAlive notifies the system that the current process is still alive.
Properties	mergeMapIntoConfig(Properties defaultConfig, Map<String,String> map)
Object	newInstance(String className) Method newInstance creates a new object instance from the given className argument delegating to any platform specific instantiation and configuration routines.
TupleEntryCollector	openSystemIntermediateForWrite()
TupleEntryIterator	openTapForRead(Tap tap) Method openTapForRead return a TupleEntryIterator for the given Tap instance.
TupleEntryCollector	openTapForWrite(Tap tap) Method openTapForWrite returns a (@link TupleCollector} for the given Tap instance.
TupleEntryCollector	openTrapForWrite(Tap trap) Method openTrapForWrite returns a (@link TupleCollector} for the given Tap instance.
void	setStatus(String status) Method setStatus is used to set the status of the current operation.
void	setStepStats(LocalStepStats stepStats)

Methods inherited from class cascading.flow.FlowProcess

getCurrentSession, getID, getIntegerProperty, getStringProperty, setCurrentSession

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LocalFlowProcess

public LocalFlowProcess()

LocalFlowProcess

public LocalFlowProcess(Properties config)

LocalFlowProcess

public LocalFlowProcess(FlowSession flowSession,
                        Properties config)

LocalFlowProcess

public LocalFlowProcess(LocalFlowProcess flowProcess,
                        Properties properties)

Method Detail

setStepStats

public void setStepStats(LocalStepStats stepStats)

getNumProcessSlices

public int getNumProcessSlices()

Description copied from class: FlowProcess

Method getNumProcessSlices returns the number of parallel slices or tasks allocated for this process execution.

For MapReduce platforms, this is the same as the number of tasks for a given MapReduce job.

Specified by:

getNumProcessSlices in class FlowProcess<Properties>

Returns:

an int

getCurrentSliceNum

public int getCurrentSliceNum()

Description copied from class: FlowProcess

Method getCurrentSliceNum returns an integer representing which slice instance currently running.

0 (zero) is the first slice instance.

Specified by:

getCurrentSliceNum in class FlowProcess<Properties>

Returns:

an int

getProperty

public Object getProperty(String key)

Description copied from class: FlowProcess

Method getProperty should be used to return configuration parameters from the underlying system.

In the case of Hadoop, the current Configuration will be queried.

Specified by:

getProperty in class FlowProcess<Properties>

Parameters:

key - of type String

Returns:

an Object

getPropertyKeys

public Collection<String> getPropertyKeys()

Description copied from class: FlowProcess

Method getPropertyKeys returns an immutable collection of all available property key values.

Specified by:

getPropertyKeys in class FlowProcess<Properties>

Returns:

a Collection

newInstance

public Object newInstance(String className)

Description copied from class: FlowProcess

Method newInstance creates a new object instance from the given className argument delegating to any platform specific instantiation and configuration routines.

Specified by:

newInstance in class FlowProcess<Properties>

Returns:

an instance of className

keepAlive

public void keepAlive()

Description copied from class: FlowProcess

Method keepAlive notifies the system that the current process is still alive. Use this method if a particular Operation takes some moments to complete. Each system is different, so calling ping every few seconds to every minute or so would be best.

This method will fail silently if the underlying mechanism to notify keepAlive status are not initialized.

Specified by:

keepAlive in class FlowProcess<Properties>

increment

public void increment(Enum counter,
                      long amount)

Description copied from class: FlowProcess

Method increment is used to increment a custom counter. Counters must be of type Enum. The amount to increment must be a integer value.

This method will fail if the underlying counter infrastructure is unavailable. See FlowProcess.isCounterStatusInitialized().

Specified by:

increment in class FlowProcess<Properties>

Parameters:

counter - of type Enum

amount - of type int

increment

public void increment(String group,
                      String counter,
                      long amount)

Description copied from class: FlowProcess

Method increment is used to increment a custom counter. The amount to increment must be a integer value.

This method will fail if the underlying counter infrastructure is unavailable. See FlowProcess.isCounterStatusInitialized().

Specified by:

increment in class FlowProcess<Properties>

Parameters:

group - of type String

counter - of type String

amount - of type int

setStatus

public void setStatus(String status)

Description copied from class: FlowProcess

Method setStatus is used to set the status of the current operation.

This method will fail if the underlying counter infrastructure is unavailable. See FlowProcess.isCounterStatusInitialized().

Specified by:

setStatus in class FlowProcess<Properties>

Parameters:

status - of type String

isCounterStatusInitialized

public boolean isCounterStatusInitialized()

Description copied from class: FlowProcess

Method isCounterStatusInitialized returns true if it is safe to increment a counter or set a status.

Specified by:

isCounterStatusInitialized in class FlowProcess<Properties>

Returns:

boolean

openTapForRead

public TupleEntryIterator openTapForRead(Tap tap)
                                  throws IOException

Description copied from class: FlowProcess

Method openTapForRead return a TupleEntryIterator for the given Tap instance.

Note the returned iterator will return the same instance of TupleEntry on every call, thus a copy must be made of either the TupleEntry or the underlying Tuple instance if they are to be stored in a Collection.

Specified by:

openTapForRead in class FlowProcess<Properties>

Parameters:

tap - of type Tap

Returns:

TupleIterator

Throws:

IOException - when there is a failure opening the resource

openTapForWrite

public TupleEntryCollector openTapForWrite(Tap tap)
                                    throws IOException

Description copied from class: FlowProcess

Method openTapForWrite returns a (@link TupleCollector} for the given Tap instance.

Specified by:

openTapForWrite in class FlowProcess<Properties>

Parameters:

tap - of type Tap

Returns:

TupleCollector

Throws:

IOException - when there is a failure opening the resource

openTrapForWrite

public TupleEntryCollector openTrapForWrite(Tap trap)
                                     throws IOException

Description copied from class: FlowProcess

Method openTrapForWrite returns a (@link TupleCollector} for the given Tap instance.

Specified by:

openTrapForWrite in class FlowProcess<Properties>

Parameters:

trap - of type Tap

Returns:

TupleCollector

Throws:

IOException - when there is a failure opening the resource

openSystemIntermediateForWrite

public TupleEntryCollector openSystemIntermediateForWrite()
                                                   throws IOException

Specified by:

openSystemIntermediateForWrite in class FlowProcess<Properties>

Throws:

IOException

copyWith

public FlowProcess copyWith(Properties object)

Specified by:

copyWith in class FlowProcess<Properties>

getConfigCopy

public Properties getConfigCopy()

Specified by:

getConfigCopy in class FlowProcess<Properties>

copyConfig

public Properties copyConfig(Properties config)

Specified by:

copyConfig in class FlowProcess<Properties>

diffConfigIntoMap

public Map<String,String> diffConfigIntoMap(Properties defaultConfig,
                                            Properties updatedConfig)

Specified by:

diffConfigIntoMap in class FlowProcess<Properties>

mergeMapIntoConfig

public Properties mergeMapIntoConfig(Properties defaultConfig,
                                     Map<String,String> map)

Specified by:

mergeMapIntoConfig in class FlowProcess<Properties>