|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object cascading.flow.BaseFlow<JobConf> cascading.flow.hadoop.HadoopFlow
public class HadoopFlow
Class HadoopFlow is the Apache Hadoop specific implementation of a Flow
.
HadoopFlowConnector
instance.
HadoopFlowConnector
Nested Class Summary |
---|
Nested classes/interfaces inherited from class cascading.flow.BaseFlow |
---|
BaseFlow.FlowHolder |
Field Summary |
---|
Fields inherited from class cascading.flow.BaseFlow |
---|
flowStats, sinks, sources, stop, stopJobsOnExit, thread |
Fields inherited from interface cascading.flow.Flow |
---|
CASCADING_FLOW_ID |
Constructor Summary | |
---|---|
protected |
HadoopFlow()
|
|
HadoopFlow(Map<Object,Object> properties,
JobConf jobConf,
FlowDef flowDef)
|
protected |
HadoopFlow(Map<Object,Object> properties,
JobConf jobConf,
String name)
|
Method Summary | |
---|---|
JobConf |
getConfig()
Method getConfig returns the internal configuration object. |
Map<Object,Object> |
getConfigAsProperties()
Method getConfiAsProperties converts the internal configuration object into a Map of
key value pairs. |
JobConf |
getConfigCopy()
Method getConfigCopy returns a copy of the internal configuration object. |
FlowProcess<JobConf> |
getFlowProcess()
|
protected int |
getMaxNumParallelSteps()
|
String |
getProperty(String key)
Method getProperty returns the value associated with the given key from the underlying properties system. |
protected void |
initConfig(Map<Object,Object> properties,
JobConf parentConfig)
This method creates a new internal Config with the parentConfig as defaults using the properties to override the defaults. |
protected void |
initFromProperties(Map<Object,Object> properties)
|
protected void |
internalClean(boolean force)
|
protected void |
internalShutdown()
|
protected void |
internalStart()
|
boolean |
isPreserveTemporaryFiles()
Method isPreserveTemporaryFiles returns false if temporary files will be cleaned when this Flow completes. |
protected JobConf |
newConfig(JobConf defaultConfig)
|
protected void |
setConfigProperty(JobConf config,
Object key,
Object value)
|
boolean |
stepsAreLocal()
Method jobsAreLocal returns true if all jobs are executed in-process as a single map and reduce task. |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait |
Constructor Detail |
---|
protected HadoopFlow()
protected HadoopFlow(Map<Object,Object> properties, JobConf jobConf, String name)
public HadoopFlow(Map<Object,Object> properties, JobConf jobConf, FlowDef flowDef)
Method Detail |
---|
protected void initFromProperties(Map<Object,Object> properties)
initFromProperties
in class BaseFlow<JobConf>
protected void initConfig(Map<Object,Object> properties, JobConf parentConfig)
BaseFlow
initConfig
in class BaseFlow<JobConf>
protected void setConfigProperty(JobConf config, Object key, Object value)
setConfigProperty
in class BaseFlow<JobConf>
protected JobConf newConfig(JobConf defaultConfig)
newConfig
in class BaseFlow<JobConf>
public JobConf getConfig()
Flow
FlowConnector
for setting
default properties visible to children. Or see FlowStepStrategy
for setting properties on
individual steps before they are executed.
public JobConf getConfigCopy()
Flow
public Map<Object,Object> getConfigAsProperties()
Flow
Map
of
key value pairs.
public String getProperty(String key)
key
- of type String
public FlowProcess<JobConf> getFlowProcess()
public boolean isPreserveTemporaryFiles()
protected void internalStart()
internalStart
in class BaseFlow<JobConf>
public boolean stepsAreLocal()
Flow
protected void internalClean(boolean force)
internalClean
in class BaseFlow<JobConf>
protected void internalShutdown()
internalShutdown
in class BaseFlow<JobConf>
protected int getMaxNumParallelSteps()
getMaxNumParallelSteps
in class BaseFlow<JobConf>
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |