cascading.flow.hadoop
Class ProcessFlow<P>

java.lang.Object
  cascading.flow.BaseFlow<JobConf>
      cascading.flow.hadoop.HadoopFlow
          cascading.flow.hadoop.ProcessFlow<P>

All Implemented Interfaces:

cascading.flow.Flow<JobConf>, cascading.management.UnitOfWork<cascading.stats.FlowStats>

public class ProcessFlow<P>
extends HadoopFlow

Class ProcessFlow is a Flow subclass that supports custom Riffle jobs.

Use this class to allow custom Riffle jobs to participate in the Cascade scheduler. If other Flow instances in the Cascade share resources with this Flow instance, all participants will be scheduled according to their dependencies (topologically).

Though this class sub-classes HadoopFlow, it does not support all the methods available or features.

Currently FlowListeners are supported but the FlowListener.onThrowable(cascading.flow.Flow, Throwable) event is not.

Nested Class Summary

Nested classes/interfaces inherited from class cascading.flow.BaseFlow

cascading.flow.BaseFlow.FlowHolder

Field Summary

Fields inherited from class cascading.flow.BaseFlow

flowStats, sinks, sources, stop, stopJobsOnExit, thread

Fields inherited from interface cascading.flow.Flow

CASCADING_FLOW_ID

Constructor Summary

ProcessFlow(Map<Object,Object> properties, String name, P process)
Constructor ProcessFlow creates a new ProcessFlow instance.

ProcessFlow(String name, P process)
Constructor ProcessFlow creates a new ProcessFlow instance.

Method Summary

void	cleanup()
void	complete()
P	getProcess() Method getProcess returns the process of this ProcessFlow object.
void	prepare()
void	setTapFromProcess() Method setTapFromProcess build Tap instance for the give process incoming and outgoing dependencies.
void	start()
void	stop()
String	toString()

Methods inherited from class cascading.flow.hadoop.HadoopFlow

getConfig, getConfigAsProperties, getConfigCopy, getFlowProcess, getMaxNumParallelSteps, getProperty, initConfig, initFromProperties, internalClean, internalShutdown, internalStart, isPreserveTemporaryFiles, newConfig, setConfigProperty, stepsAreLocal

Methods inherited from class cascading.flow.BaseFlow

addListener, addStepListener, areSinksStale, areSourcesNewer, createConfig, createFlowThread, deleteCheckpointsIfNotUpdate, deleteCheckpointsIfReplace, deleteSinks, deleteSinksIfNotUpdate, deleteSinksIfReplace, deleteTrapsIfNotUpdate, deleteTrapsIfReplace, fireOnCompleted, fireOnStarting, fireOnStopping, fireOnThrowable, getCascadeID, getCascadingServices, getCheckpointNames, getCheckpoints, getCheckpointsCollection, getClassPath, getFieldsFor, getFlowSession, getFlowSkipStrategy, getFlowStats, getFlowSteps, getFlowStepStrategy, getHolder, getID, getName, getPlatformInfo, getRunID, getSink, getSink, getSinkModified, getSinkNames, getSinks, getSinksCollection, getSource, getSourceNames, getSources, getSourcesCollection, getSpawnStrategy, getStats, getSubmitPriority, getTags, getTrapNames, getTraps, getTrapsCollection, handleExecutorShutdown, hasListeners, hasStepListeners, initialize, initializeNewJobsMap, initSteps, internalStopAllJobs, isSkipFlow, isStopJobsOnExit, logInfo, openSink, openSink, openSource, openSource, openTapForRead, openTapForWrite, openTrap, openTrap, presentSinkFields, presentSourceFields, registerShutdownHook, removeListener, removeStepListener, resourceExists, retrieveSinkFields, retrieveSourceFields, setCascade, setCheckpoints, setFlowSkipStrategy, setFlowStepGraph, setFlowStepStrategy, setName, setSinks, setSources, setSpawnStrategy, setSubmitPriority, setTraps, updateSchemes, writeDOT, writeStepsDOT

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

ProcessFlow

@ConstructorProperties(value={"name","process"})
public ProcessFlow(String name,
                                              P process)

Constructor ProcessFlow creates a new ProcessFlow instance.

Parameters:

name - of type String

process - of type JobConf

ProcessFlow

@ConstructorProperties(value={"properties","name","process"})
public ProcessFlow(Map<Object,Object> properties,
                                              String name,
                                              P process)

Constructor ProcessFlow creates a new ProcessFlow instance.

Parameters:

properties - of type Map

name - of type String

process - of type P

Method Detail

setTapFromProcess

public void setTapFromProcess()

Method setTapFromProcess build Tap instance for the give process incoming and outgoing dependencies.

This method may be called repeatedly to re-configure the source and sink taps.

getProcess

public P getProcess()

Method getProcess returns the process of this ProcessFlow object.

Returns:

the process (type P) of this ProcessFlow object.

prepare

public void prepare()

Specified by:

prepare in interface cascading.flow.Flow<JobConf>

Specified by:

prepare in interface cascading.management.UnitOfWork<cascading.stats.FlowStats>

Overrides:

prepare in class cascading.flow.BaseFlow<JobConf>

start

public void start()

Specified by:

start in interface cascading.flow.Flow<JobConf>

Specified by:

start in interface cascading.management.UnitOfWork<cascading.stats.FlowStats>

Overrides:

start in class cascading.flow.BaseFlow<JobConf>

stop

public void stop()

Specified by:

stop in interface cascading.flow.Flow<JobConf>

Specified by:

stop in interface cascading.management.UnitOfWork<cascading.stats.FlowStats>

Overrides:

stop in class cascading.flow.BaseFlow<JobConf>

complete

public void complete()

Specified by:

complete in interface cascading.flow.Flow<JobConf>

Specified by:

complete in interface cascading.management.UnitOfWork<cascading.stats.FlowStats>

Overrides:

complete in class cascading.flow.BaseFlow<JobConf>

cleanup

public void cleanup()

Specified by:

cleanup in interface cascading.flow.Flow<JobConf>

Specified by:

cleanup in interface cascading.management.UnitOfWork<cascading.stats.FlowStats>

Overrides:

cleanup in class cascading.flow.BaseFlow<JobConf>

toString

public String toString()

Overrides:

toString in class cascading.flow.BaseFlow<JobConf>