cascading.flow
Class MapReduceFlow

java.lang.Object
  cascading.flow.Flow
      cascading.flow.MapReduceFlow

All Implemented Interfaces:

Runnable

public class MapReduceFlow
extends Flow

Class MapReduceFlow is a Flow subclass that supports custom MapReduce jobs pre-configured via the JobConf object.

Use this class to allow custom MapReduce jobs to participate in the Cascade scheduler. If other Flow instances in the Cascade share resources with this Flow instance, all participants will be scheduled according to their dependencies (topologically).

Set the parameter deleteSinkOnInit to true if the outputPath in the jobConf should be deleted before executing the MapReduce job.

Nested Class Summary

Nested classes/interfaces inherited from class cascading.flow.Flow

Flow.FlowHolder

Field Summary

Fields inherited from class cascading.flow.Flow

sources, stopJobsOnExit

Constructor Summary

MapReduceFlow(JobConf jobConf)
Constructor MapReduceFlow creates a new MapReduceFlow instance.

MapReduceFlow(JobConf jobConf, boolean deleteSinkOnInit)
Constructor MapReduceFlow creates a new MapReduceFlow instance.

MapReduceFlow(String name, JobConf jobConf)
Constructor MapReduceFlow creates a new MapReduceFlow instance.

MapReduceFlow(String name, JobConf jobConf, boolean deleteSinkOnInit)
Constructor MapReduceFlow creates a new MapReduceFlow instance.

MapReduceFlow(String name, JobConf jobConf, boolean deleteSinkOnInit, boolean stopJobsOnExit)
Constructor MapReduceFlow creates a new MapReduceFlow instance.

Method Summary

Methods inherited from class cascading.flow.Flow

addListener, areSinksStale, areSourcesNewer, cleanup, complete, deleteSinks, deleteSinksIfNotAppend, deleteSinksIfNotUpdate, getFlowSkipStrategy, getFlowStats, getHolder, getID, getJobConf, getJobPollingInterval, getJobPollingInterval, getName, getPreserveTemporaryFiles, getProperty, getSink, getSinkModified, getSinks, getSinksCollection, getSources, getSourcesCollection, getSteps, getStopJobsOnExit, getSubmitPriority, getTraps, getTrapsCollection, hasListeners, isPreserveTemporaryFiles, isSkipFlow, isStopJobsOnExit, jobsAreLocal, openSink, openSink, openSource, openSource, openTapForRead, openTapForWrite, openTrap, openTrap, prepare, removeListener, run, setFlowSkipStrategy, setJobPollingInterval, setName, setPreserveTemporaryFiles, setProperty, setSinks, setSources, setStepGraph, setStopJobsOnExit, setSubmitPriority, setTraps, start, stop, tapPathExists, toString, writeDOT, writeStepsDOT

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

MapReduceFlow

@ConstructorProperties(value="jobConf")
public MapReduceFlow(JobConf jobConf)

Constructor MapReduceFlow creates a new MapReduceFlow instance.

Parameters:

jobConf - of type JobConf

MapReduceFlow

@ConstructorProperties(value={"jobConf","deleteSinkOnInit"})
public MapReduceFlow(JobConf jobConf,
                                                boolean deleteSinkOnInit)

Constructor MapReduceFlow creates a new MapReduceFlow instance.

Parameters:

jobConf - of type JobConf

deleteSinkOnInit - of type boolean

MapReduceFlow

@ConstructorProperties(value={"name","jobConf"})
public MapReduceFlow(String name,
                                                JobConf jobConf)

Constructor MapReduceFlow creates a new MapReduceFlow instance.

Parameters:

name - of type String

jobConf - of type JobConf

MapReduceFlow

@ConstructorProperties(value={"name","jobConf","deleteSinkOnInit"})
public MapReduceFlow(String name,
                                                JobConf jobConf,
                                                boolean deleteSinkOnInit)

Constructor MapReduceFlow creates a new MapReduceFlow instance.

Parameters:

name - of type String

jobConf - of type JobConf

deleteSinkOnInit - of type boolean

MapReduceFlow

@ConstructorProperties(value={"name","jobConf","deleteSinkOnInit","stopJobsOnExit"})
public MapReduceFlow(String name,
                                                JobConf jobConf,
                                                boolean deleteSinkOnInit,
                                                boolean stopJobsOnExit)

Constructor MapReduceFlow creates a new MapReduceFlow instance.

Parameters:

name - of type String

jobConf - of type JobConf

deleteSinkOnInit - of type boolean

stopJobsOnExit - of type boolean