|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object cascading.cascade.Cascade
public class Cascade
A Cascade is an assembly of Flow
instances that share or depend on equivalent Tap
instances and are executed as
a single group. The most common case is where one Flow instance depends on a Tap created by a second Flow instance. This
dependency chain can continue as practical.
Tap
is newer than
a subsequent sink Tap
in the assembly, the connecting Flow
(s) will be executed
when the Cascade executed. If all the targets (sinks) are up to date, the Cascade exits immediately and does nothing.
The concept of 'stale' is pluggable, see the FlowSkipStrategy
class.
When a Cascade starts up, if first verifies which Flow instances have stale sinks, if the sinks are not stale, the
method BaseFlow.deleteSinksIfNotUpdate()
is called. Before appends were supported (logically)
the Cascade deleted all the sinks in a Flow.
The new consequence of this is if the Cascade fails, but does complete a Flow that appended or updated data, re-running
the Cascade (and the successful append/update Flow) will re-update data to the source. Some systems may be idempotent and
may not have any side-effects. So plan accordingly.
Flow
,
FlowSkipStrategy
Nested Class Summary | |
---|---|
protected class |
Cascade.CascadeJob
Class CascadeJob manages Flow execution in the current Cascade instance. |
Method Summary | |
---|---|
void |
cleanup()
|
void |
complete()
Method complete begins the current Cascade process if method start() was not previously called. |
List<Flow> |
findFlows(String regex)
Method findFlows returns a List of flows whose names match the given regex pattern. |
Collection<Flow> |
findFlowsSinkingTo(String identifier)
Method findFlowsSinkingTo returns all Flow instances that writes to a sink with the given identifier. |
Collection<Flow> |
findFlowsSourcingFrom(String identifier)
Method findFlowsSourcingFrom returns all Flow instances that reads from a source with the given identifier. |
CascadeStats |
getCascadeStats()
Method getCascadeStats returns the cascadeStats of this Cascade object. |
protected cascading.cascade.planner.FlowGraph |
getFlowGraph()
|
List<Flow> |
getFlows()
Method getFlows returns the flows managed by this Cascade object. |
FlowSkipStrategy |
getFlowSkipStrategy()
Method getFlowSkipStrategy returns the current FlowSkipStrategy used by this Flow. |
String |
getID()
Method getID returns the ID of this Cascade object. |
String |
getName()
Method getName returns the name of this Cascade object. |
Collection<Flow> |
getPredecessorFlows(Flow flow)
Method getPredecessorFlows returns a Collection of all the Flow instances that will be executed before the given Flow instance. |
UnitOfWorkSpawnStrategy |
getSpawnStrategy()
|
CascadeStats |
getStats()
|
Collection<Flow> |
getSuccessorFlows(Flow flow)
Method getSuccessorFlows returns a Collection of all the Flow instances that will be executed after the given Flow instance. |
String |
getTags()
Method getTags returns the tags associated with this Cascade object. |
protected cascading.cascade.planner.TapGraph |
getTapGraph()
|
void |
prepare()
|
protected void |
printElementGraph(String filename,
org.jgrapht.graph.SimpleDirectedGraph<String,BaseFlow.FlowHolder> graph)
|
FlowSkipStrategy |
setFlowSkipStrategy(FlowSkipStrategy flowSkipStrategy)
Method setFlowSkipStrategy sets a new FlowSkipStrategy , the current strategy, if any, is returned. |
void |
setSpawnStrategy(UnitOfWorkSpawnStrategy spawnStrategy)
|
void |
start()
Method start begins the current Cascade process. |
void |
stop()
|
String |
toString()
|
void |
writeDOT(String filename)
Method writeDOT writes this element graph to a DOT file for easy visualization and debugging. |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait |
Method Detail |
---|
public String getName()
getName
in interface UnitOfWork<CascadeStats>
public String getID()
getID
in interface UnitOfWork<CascadeStats>
public String getTags()
getTags
in interface UnitOfWork<CascadeStats>
public CascadeStats getCascadeStats()
public CascadeStats getStats()
getStats
in interface UnitOfWork<CascadeStats>
protected cascading.cascade.planner.FlowGraph getFlowGraph()
protected cascading.cascade.planner.TapGraph getTapGraph()
public List<Flow> getFlows()
Flow
instances
will be in topological order.
public List<Flow> findFlows(String regex)
regex
- of type String
public Collection<Flow> getSuccessorFlows(Flow flow)
flow
- of type Flow
public Collection<Flow> getPredecessorFlows(Flow flow)
flow
- of type Flow
public Collection<Flow> findFlowsSourcingFrom(String identifier)
identifier
- of type String
public Collection<Flow> findFlowsSinkingTo(String identifier)
identifier
- of type String
public FlowSkipStrategy getFlowSkipStrategy()
FlowSkipStrategy
used by this Flow.
public FlowSkipStrategy setFlowSkipStrategy(FlowSkipStrategy flowSkipStrategy)
FlowSkipStrategy
, the current strategy, if any, is returned.
If a strategy is given, it will be used as the strategy for all BaseFlow
instances managed by this Cascade instance.
To revert back to consulting the strategies associated with each Flow instance, re-set this value to null
, its
default value.
FlowSkipStrategy instances define when a Flow instance should be skipped. The default strategy is FlowSkipIfSinkNotStale
and is inherited from the Flow instance in question. An alternative strategy would be FlowSkipIfSinkExists
.
A FlowSkipStrategy will not be consulted when executing a Flow directly through start()
flowSkipStrategy
- of type FlowSkipStrategy
public void prepare()
prepare
in interface UnitOfWork<CascadeStats>
public void start()
complete()
to block
until the Cascade completes.
start
in interface UnitOfWork<CascadeStats>
public void complete()
start()
was not previously called. This method
blocks until the process completes.
complete
in interface UnitOfWork<CascadeStats>
RuntimeException
- wrapping any exception thrown internally.public void stop()
stop
in interface UnitOfWork<CascadeStats>
public void cleanup()
cleanup
in interface UnitOfWork<CascadeStats>
public void writeDOT(String filename)
filename
- of type Stringprotected void printElementGraph(String filename, org.jgrapht.graph.SimpleDirectedGraph<String,BaseFlow.FlowHolder> graph)
public String toString()
toString
in class Object
public UnitOfWorkSpawnStrategy getSpawnStrategy()
getSpawnStrategy
in interface UnitOfWork<CascadeStats>
public void setSpawnStrategy(UnitOfWorkSpawnStrategy spawnStrategy)
setSpawnStrategy
in interface UnitOfWork<CascadeStats>
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |