cascading.flow.hadoop
Class HadoopFlowConnector

java.lang.Object
  cascading.flow.FlowConnector
      cascading.flow.hadoop.HadoopFlowConnector

public class HadoopFlowConnector
extends FlowConnector

Use the HadoopFlowConnector to link source and sink Tap instances with an assembly of Pipe instances into an executable HadoopFlow for execution on an Apache Hadoop cluster.

See Also:

AppProps, FlowConnectorProps, FlowDef, MapReduceFlow

Field Summary

Fields inherited from class cascading.flow.FlowConnector

properties

Constructor Summary

HadoopFlowConnector()
Constructor FlowConnector creates a new FlowConnector instance.

HadoopFlowConnector(Map<Object,Object> properties)
Constructor FlowConnector creates a new FlowConnector instance using the given Properties instance as default value for the underlying jobs.

Method Summary

protected cascading.flow.planner.FlowPlanner	createFlowPlanner()
protected Class<? extends Scheme>	getDefaultIntermediateSchemeClass()

Methods inherited from class cascading.flow.FlowConnector

connect, connect, connect, connect, connect, connect, connect, connect, connect, connect, connect, connect, connect, connect, connect, getIntermediateSchemeClass, getPlatformInfo, getProperties, setApplicationJarClass, setApplicationJarPath

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HadoopFlowConnector

public HadoopFlowConnector()

Constructor FlowConnector creates a new FlowConnector instance.

All properties passed to Hadoop are retrieved from a default instantiation of the Hadoop JobConf which pulls all properties from the local CLASSPATH.

HadoopFlowConnector

@ConstructorProperties(value="properties")
public HadoopFlowConnector(Map<Object,Object> properties)

Constructor FlowConnector creates a new FlowConnector instance using the given Properties instance as default value for the underlying jobs. All properties are copied to a new native configuration instance.

Parameters:

properties - of type Properties

Method Detail

getDefaultIntermediateSchemeClass

protected Class<? extends Scheme> getDefaultIntermediateSchemeClass()

Specified by:

getDefaultIntermediateSchemeClass in class FlowConnector

createFlowPlanner

protected cascading.flow.planner.FlowPlanner createFlowPlanner()

Specified by:

createFlowPlanner in class FlowConnector