cascading.flow.hadoop
Class ProcessFlow<P>

java.lang.Object
  extended by cascading.flow.BaseFlow<JobConf>
      extended by cascading.flow.hadoop.HadoopFlow
          extended by cascading.flow.hadoop.ProcessFlow<P>
All Implemented Interfaces:
Flow<JobConf>, UnitOfWork<FlowStats>

public class ProcessFlow<P>
extends HadoopFlow

Class ProcessFlow is a Flow subclass that supports custom Riffle jobs.

Use this class to allow custom Riffle jobs to participate in the Cascade scheduler. If other Flow instances in the Cascade share resources with this Flow instance, all participants will be scheduled according to their dependencies (topologically).


Nested Class Summary
 
Nested classes/interfaces inherited from class cascading.flow.BaseFlow
BaseFlow.FlowHolder
 
Field Summary
 
Fields inherited from class cascading.flow.BaseFlow
flowStats, sinks, sources, stop, stopJobsOnExit, thread
 
Fields inherited from interface cascading.flow.Flow
CASCADING_FLOW_ID
 
Constructor Summary
ProcessFlow(Map<Object,Object> properties, String name, P process)
          Constructor ProcessFlow creates a new ProcessFlow instance.
ProcessFlow(String name, P process)
          Constructor ProcessFlow creates a new ProcessFlow instance.
 
Method Summary
 void cleanup()
           
 void complete()
          Method complete starts the current Flow instance if it has not be previously started, then block until completion.
 P getProcess()
          Method getProcess returns the process of this ProcessFlow object.
 void prepare()
          Method prepare is used by a Cascade to notify the given Flow it should initialize or clear any resources necessary for Flow.start() to be called successfully.
 void setTapFromProcess()
          Method setTapFromProcess build Tap instance for the give process incoming and outgoing dependencies.
 void start()
          Method start begins the execution of this Flow instance.
 void stop()
          Method stop stops all running jobs, killing any currently executing.
 String toString()
           
 
Methods inherited from class cascading.flow.hadoop.HadoopFlow
getConfig, getConfigAsProperties, getConfigCopy, getFlowProcess, getMaxNumParallelSteps, getProperty, initConfig, initFromProperties, internalClean, internalShutdown, internalStart, isPreserveTemporaryFiles, newConfig, setConfigProperty, stepsAreLocal
 
Methods inherited from class cascading.flow.BaseFlow
addListener, areSinksStale, areSourcesNewer, createConfig, deleteCheckpointsIfNotUpdate, deleteCheckpointsIfReplace, deleteSinks, deleteSinksIfNotUpdate, deleteSinksIfReplace, deleteTrapsIfNotUpdate, deleteTrapsIfReplace, fireOnStarting, fireOnStopping, getCascadeID, getCascadingServices, getCheckpoints, getCheckpointsCollection, getFieldsFor, getFlowSession, getFlowSkipStrategy, getFlowStats, getFlowSteps, getFlowStepStrategy, getHolder, getID, getName, getSink, getSink, getSinkModified, getSinks, getSinksCollection, getSource, getSources, getSourcesCollection, getSpawnStrategy, getStats, getSubmitPriority, getTags, getTraps, getTrapsCollection, handleExecutorShutdown, hasListeners, initialize, initializeNewJobsMap, initSteps, internalStopAllJobs, isSkipFlow, isStopJobsOnExit, logInfo, openSink, openSink, openSource, openSource, openTapForRead, openTapForWrite, openTrap, openTrap, presentSinkFields, presentSourceFields, registerShutdownHook, removeListener, resourceExists, retrieveSinkFields, retrieveSourceFields, setCascade, setCheckpoints, setFlowSkipStrategy, setFlowStepGraph, setFlowStepStrategy, setName, setSinks, setSources, setSpawnStrategy, setSubmitPriority, setTraps, updateSchemes, writeDOT, writeStepsDOT
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Constructor Detail

ProcessFlow

@ConstructorProperties(value={"name","process"})
public ProcessFlow(String name,
                                              P process)
Constructor ProcessFlow creates a new ProcessFlow instance.

Parameters:
name - of type String
process - of type JobConf

ProcessFlow

@ConstructorProperties(value={"properties","name","process"})
public ProcessFlow(Map<Object,Object> properties,
                                              String name,
                                              P process)
Constructor ProcessFlow creates a new ProcessFlow instance.

Parameters:
properties - of type Map
name - of type String
process - of type P
Method Detail

setTapFromProcess

public void setTapFromProcess()
Method setTapFromProcess build Tap instance for the give process incoming and outgoing dependencies.

This method may be called repeatedly to re-configure the source and sink taps.


getProcess

public P getProcess()
Method getProcess returns the process of this ProcessFlow object.

Returns:
the process (type P) of this ProcessFlow object.

prepare

public void prepare()
Description copied from interface: Flow
Method prepare is used by a Cascade to notify the given Flow it should initialize or clear any resources necessary for Flow.start() to be called successfully.

Specifically, this implementation calls BaseFlow.deleteSinksIfNotUpdate() && BaseFlow.deleteTrapsIfNotUpdate().

Specified by:
prepare in interface Flow<JobConf>
Specified by:
prepare in interface UnitOfWork<FlowStats>
Overrides:
prepare in class BaseFlow<JobConf>

start

public void start()
Description copied from interface: Flow
Method start begins the execution of this Flow instance. It will return immediately. Use the method Flow.complete() to block until this Flow completes.

Specified by:
start in interface Flow<JobConf>
Specified by:
start in interface UnitOfWork<FlowStats>
Overrides:
start in class BaseFlow<JobConf>

stop

public void stop()
Description copied from interface: Flow
Method stop stops all running jobs, killing any currently executing.

Specified by:
stop in interface Flow<JobConf>
Specified by:
stop in interface UnitOfWork<FlowStats>
Overrides:
stop in class BaseFlow<JobConf>

complete

public void complete()
Description copied from interface: Flow
Method complete starts the current Flow instance if it has not be previously started, then block until completion.

Specified by:
complete in interface Flow<JobConf>
Specified by:
complete in interface UnitOfWork<FlowStats>
Overrides:
complete in class BaseFlow<JobConf>

cleanup

public void cleanup()
Specified by:
cleanup in interface Flow<JobConf>
Specified by:
cleanup in interface UnitOfWork<FlowStats>
Overrides:
cleanup in class BaseFlow<JobConf>

toString

public String toString()
Overrides:
toString in class BaseFlow<JobConf>


Copyright © 2007-2012 Concurrent, Inc. All Rights Reserved.