cascading.flow.hadoop.planner
Class HadoopFlowStepJob

java.lang.Object
  cascading.flow.planner.FlowStepJob<JobConf>
      cascading.flow.hadoop.planner.HadoopFlowStepJob

All Implemented Interfaces:

Callable<Throwable>

public class HadoopFlowStepJob
extends cascading.flow.planner.FlowStepJob<JobConf>

Field Summary

Fields inherited from class cascading.flow.planner.FlowStepJob

flowStep, flowStepStats, pollingInterval, predecessors, statsStoreInterval, stepName, throwable

Constructor Summary

HadoopFlowStepJob(cascading.management.state.ClientState clientState, cascading.flow.planner.BaseFlowStep flowStep, JobConf currentConf)

Method Summary

protected cascading.stats.FlowStepStats	createStepStats(cascading.management.state.ClientState clientState)
protected void	dumpDebugInfo()
JobConf	getConfig()
static long	getJobPollingInterval(JobConf jobConf)
protected Throwable	getThrowable()
protected void	internalBlockOnStop()
protected boolean	internalIsStarted()
protected String	internalJobId()
protected boolean	internalNonBlockingIsComplete()
protected boolean	internalNonBlockingIsSuccessful()
protected void	internalNonBlockingStart()
protected boolean	isRemoteExecution()
static void	reportLocalError(Throwable throwable) Internal method to report errors that happen on hadoop local mode.

Methods inherited from class cascading.flow.planner.FlowStepJob

blockOnJob, blockOnPredecessors, blockTillCompleteOrStopped, call, getStepStats, isSkipFlowStep, isStarted, isSuccessful, setPredecessors, sleepForPollingInterval, start, stop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HadoopFlowStepJob

public HadoopFlowStepJob(cascading.management.state.ClientState clientState,
                         cascading.flow.planner.BaseFlowStep flowStep,
                         JobConf currentConf)

Method Detail

getJobPollingInterval

public static long getJobPollingInterval(JobConf jobConf)

getConfig

public JobConf getConfig()

Specified by:

getConfig in class cascading.flow.planner.FlowStepJob<JobConf>

createStepStats

protected cascading.stats.FlowStepStats createStepStats(cascading.management.state.ClientState clientState)

Specified by:

createStepStats in class cascading.flow.planner.FlowStepJob<JobConf>

internalBlockOnStop

protected void internalBlockOnStop()
                            throws IOException

Specified by:

internalBlockOnStop in class cascading.flow.planner.FlowStepJob<JobConf>

Throws:

IOException

internalNonBlockingStart

protected void internalNonBlockingStart()
                                 throws IOException

Specified by:

internalNonBlockingStart in class cascading.flow.planner.FlowStepJob<JobConf>

Throws:

IOException

internalNonBlockingIsSuccessful

protected boolean internalNonBlockingIsSuccessful()
                                           throws IOException

Specified by:

internalNonBlockingIsSuccessful in class cascading.flow.planner.FlowStepJob<JobConf>

Throws:

IOException

isRemoteExecution

protected boolean isRemoteExecution()

Specified by:

isRemoteExecution in class cascading.flow.planner.FlowStepJob<JobConf>

getThrowable

protected Throwable getThrowable()

Specified by:

getThrowable in class cascading.flow.planner.FlowStepJob<JobConf>

internalJobId

protected String internalJobId()

Specified by:

internalJobId in class cascading.flow.planner.FlowStepJob<JobConf>

internalNonBlockingIsComplete

protected boolean internalNonBlockingIsComplete()
                                         throws IOException

Specified by:

internalNonBlockingIsComplete in class cascading.flow.planner.FlowStepJob<JobConf>

Throws:

IOException

dumpDebugInfo

protected void dumpDebugInfo()

Specified by:

dumpDebugInfo in class cascading.flow.planner.FlowStepJob<JobConf>

internalIsStarted

protected boolean internalIsStarted()

Specified by:

internalIsStarted in class cascading.flow.planner.FlowStepJob<JobConf>

reportLocalError

public static void reportLocalError(Throwable throwable)

Internal method to report errors that happen on hadoop local mode. Hadoops local JobRunner does not give access to TaskReports, but we want to be able to capture the exception and not just print it to stderr. FlowMapper and FlowReducer use this method.

Parameters:

throwable - the throwable to be reported.