cascading.flow.hadoop
Class HadoopFlowConnector

java.lang.Object
  extended by cascading.flow.FlowConnector
      extended by cascading.flow.hadoop.HadoopFlowConnector

public class HadoopFlowConnector
extends FlowConnector

Use the HadoopFlowConnector to link source and sink Tap instances with an assembly of Pipe instances into an executable HadoopFlow for execution on an Apache Hadoop cluster.

See Also:
AppProps, FlowConnectorProps, FlowDef, MapReduceFlow

Field Summary
 
Fields inherited from class cascading.flow.FlowConnector
properties
 
Constructor Summary
HadoopFlowConnector()
          Constructor FlowConnector creates a new FlowConnector instance.
HadoopFlowConnector(Map<Object,Object> properties)
          Constructor FlowConnector creates a new FlowConnector instance using the given Properties instance as default value for the underlying jobs.
 
Method Summary
protected  FlowPlanner createFlowPlanner()
           
protected  Class<? extends Scheme> getDefaultIntermediateSchemeClass()
           
 
Methods inherited from class cascading.flow.FlowConnector
connect, connect, connect, connect, connect, connect, connect, connect, connect, connect, connect, connect, connect, connect, connect, getIntermediateSchemeClass, getPlatformInfo, getProperties, setApplicationJarClass, setApplicationJarPath
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

HadoopFlowConnector

public HadoopFlowConnector()
Constructor FlowConnector creates a new FlowConnector instance.

All properties passed to Hadoop are retrieved from a default instantiation of the Hadoop JobConf which pulls all properties from the local CLASSPATH.


HadoopFlowConnector

@ConstructorProperties(value="properties")
public HadoopFlowConnector(Map<Object,Object> properties)
Constructor FlowConnector creates a new FlowConnector instance using the given Properties instance as default value for the underlying jobs. All properties are copied to a new native configuration instance.

Parameters:
properties - of type Properties
Method Detail

getDefaultIntermediateSchemeClass

protected Class<? extends Scheme> getDefaultIntermediateSchemeClass()
Specified by:
getDefaultIntermediateSchemeClass in class FlowConnector

createFlowPlanner

protected FlowPlanner createFlowPlanner()
Specified by:
createFlowPlanner in class FlowConnector


Copyright © 2007-2015 Concurrent, Inc. All Rights Reserved.