cascading.stats.hadoop

Class BaseHadoopStepStats<JobStatusClient,Counters>

java.lang.Object

cascading.stats.CascadingStats<FlowNodeStats>

cascading.stats.FlowStepStats

cascading.stats.hadoop.BaseHadoopStepStats<JobStatusClient,Counters>

All Implemented Interfaces:

ProvidesCounters, Serializable

Direct Known Subclasses:

HadoopStepStats

public abstract class BaseHadoopStepStats<JobStatusClient,Counters>
extends FlowStepStats

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class cascading.stats.CascadingStats

CascadingStats.Status, CascadingStats.Type

Field Summary

Fields

Modifier and Type	Field and Description
protected CounterCache<JobStatusClient,Counters>	counterCache

Fields inherited from class cascading.stats.CascadingStats

clientState, STATS_COMPLETE_CHILD_DETAILS_BLOCK_DURATION, STATS_STORE_INTERVAL

Constructor Summary

Constructors

Constructor and Description
BaseHadoopStepStats(FlowStep flowStep, ClientState clientState)

Method Summary

Methods

Modifier and Type	Method and Description
protected Counters	cachedCounters(boolean force)
Collection<String>	getCounterGroups() Method getCounterGroups returns all of the Hadoop counter groups.
Collection<String>	getCounterGroupsMatching(String regex) Method getCounterGroupsMatching returns all the Hadoop counter groups that match the give regex pattern.
Collection<String>	getCountersFor(String group) Method getCountersFor returns the Hadoop counters for the given group.
long	getCounterValue(Enum counter) Method getCounterValue returns the Hadoop counter value for the given counter enum.
long	getCounterValue(String group, String counter) Method getCounterValue returns the Hadoop counter value for the given group and counter name.
abstract JobStatusClient	getJobStatusClient() Method getRunningJob returns the Hadoop RunningJob managing this Hadoop job.
long	getLastSuccessfulCounterFetchTime()
void	recordChildStats() Synchronized to prevent state changes mid record, #stop may be called out of band

Methods inherited from class cascading.stats.FlowStepStats

addNodeStats, getChildren, getChildWith, getFlowNodeIDs, getFlowNodeStats, getFlowNodeStatsMap, getFlowStep, getID, getNodesCount, getProcessLogger, getProcessStatusURL, getProcessStepID, getType, hasCapturedFinalDetail, recordInfo, toString

Methods inherited from class cascading.stats.CascadingStats

addListener, captureDetail, captureDetail, cleanup, fireListeners, getCountersFor, getCurrentDuration, getDuration, getFinishedTime, getName, getPendingTime, getRunTime, getStartTime, getStatsString, getStatus, getSubmitTime, getThrowable, getThrowableTrace, isDetailStale, isEngaged, isFailed, isFinished, isPending, isRunning, isSkipped, isStarted, isStopped, isSubmitted, isSuccessful, logDebug, logError, logError, logInfo, logWarn, markDetailCaptured, markFailed, markFailed, markFailed, markFailed, markPending, markPendingTime, markRunning, markRunTime, markSkipped, markStarted, markStartedThenRunning, markStartTime, markStartToRunTime, markStopped, markSubmitted, markSubmitTime, markSuccessful, prepare, recordStats, removeListener, setStatsStoreInterval

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

counterCache

protected CounterCache<JobStatusClient,Counters> counterCache

Constructor Detail

BaseHadoopStepStats

public BaseHadoopStepStats(FlowStep flowStep,
                   ClientState clientState)

Method Detail

getJobStatusClient

public abstract JobStatusClient getJobStatusClient()

Method getRunningJob returns the Hadoop RunningJob managing this Hadoop job.

Returns:

the runningJob (type RunningJob) of this HadoopStepStats object.

getLastSuccessfulCounterFetchTime

public long getLastSuccessfulCounterFetchTime()

getCounterGroups

public Collection<String> getCounterGroups()

Method getCounterGroups returns all of the Hadoop counter groups.

Returns:

the counterGroups (type Collection) of this HadoopStepStats object.

getCounterGroupsMatching

public Collection<String> getCounterGroupsMatching(String regex)

Method getCounterGroupsMatching returns all the Hadoop counter groups that match the give regex pattern.

Specified by:

getCounterGroupsMatching in class CascadingStats<FlowNodeStats>

Parameters:

regex - of String

Returns:

Collection

getCountersFor

public Collection<String> getCountersFor(String group)

Method getCountersFor returns the Hadoop counters for the given group.

Parameters:

group - of String

Returns:

Collection

getCounterValue

public long getCounterValue(Enum counter)

Method getCounterValue returns the Hadoop counter value for the given counter enum.

Parameters:

counter - of Enum

Returns:

long

getCounterValue

public long getCounterValue(String group,
                   String counter)

Method getCounterValue returns the Hadoop counter value for the given group and counter name.

Parameters:

group - of String

counter - of String

Returns:

long

cachedCounters

protected Counters cachedCounters(boolean force)

recordChildStats

public void recordChildStats()

Synchronized to prevent state changes mid record, #stop may be called out of band

Specified by:

recordChildStats in class FlowStepStats