org.springframework.data.hadoop.mapreduce
Class JobFactoryBean

java.lang.Object
  extended by org.springframework.data.hadoop.mapreduce.JobFactoryBean
All Implemented Interfaces:
BeanClassLoaderAware, BeanNameAware, FactoryBean<Job>, InitializingBean

public class JobFactoryBean
extends Object
implements InitializingBean, FactoryBean<Job>, BeanNameAware, BeanClassLoaderAware

Factory bean for creating a Hadoop Map-Reduce job. Note that the setters for the class access class names (String) instead of direct classes. This is done on purpose in case the job itself and its dependencies are on a jar not available on the classpath (setJar(Resource)) in which case a special, on-the-fly classloader is used.

Author:
Costin Leau

Constructor Summary
JobFactoryBean()
           
 
Method Summary
 void afterPropertiesSet()
           
 Job getObject()
           
 Class<?> getObjectType()
           
 boolean isSingleton()
           
protected  void processJob(Job job)
           
 void setArchives(Resource... archives)
          Sets the archives to be unarchive to the map reduce cluster.
 void setBeanClassLoader(ClassLoader classLoader)
           
 void setBeanName(String name)
           
 void setCodec(String codecClass)
          Sets the job codec class.
 void setCombiner(String combiner)
          Sets the job combiner class.
 void setCompressOutput(Boolean compressOutput)
          Indicates whether the job output should be compressed or not.
 void setConfiguration(Configuration configuration)
          Sets the Hadoop configuration to use.
 void setFiles(Resource... files)
          Sets the files to be copied to the map reduce cluster.
 void setGroupingComparator(String groupingComparator)
          Sets the job grouping comparator class.
 void setInputFormat(String inputFormat)
          Sets the job input format class.
 void setInputPath(String... inputPath)
          Sets the job input path.
 void setJar(Resource jar)
          Sets the job jar (which might not be on the classpath).
 void setJarByClass(Class<?> jarClass)
          Determines the job jar (available on the classpath) based on the given class.
 void setKey(String key)
          Sets the job key class.
 void setLibs(Resource... libJars)
          Sets the jar files to include in the classpath.
 void setMapKey(String mapKey)
          Sets the job map key class.
 void setMapper(String mapper)
          Sets the job mapper class.
 void setMapValue(String mapValue)
          Sets the job map value class.
 void setName(String name)
          Sets the job name.
 void setNumReduceTasks(Integer numReduceTasks)
          Sets the number of reduce task for this job.
 void setOutputFormat(String outputFormat)
          Sets the job output format class.
 void setOutputPath(String outputPath)
          Sets the job output path.
 void setPartitioner(String partitioner)
          Sets the job partitioner class.
 void setProperties(Properties properties)
          The configuration properties to set for this job.
 void setReducer(String reducer)
          Sets the job reducer class.
 void setSortComparator(String sortComparator)
          Sets the job sort comparator class.
 void setUser(String user)
          Sets the user impersonation (optional) for running this job.
 void setValidatePaths(Boolean validatePaths)
          Indicates whether the job input/output paths should be validated (default) before the job is submitted.
 void setValue(String value)
          Sets the job value class.
 void setWorkingDir(String workingDir)
          Sets the job working directory.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

JobFactoryBean

public JobFactoryBean()
Method Detail

setBeanClassLoader

public void setBeanClassLoader(ClassLoader classLoader)
Specified by:
setBeanClassLoader in interface BeanClassLoaderAware

setBeanName

public void setBeanName(String name)
Specified by:
setBeanName in interface BeanNameAware

getObject

public Job getObject()
              throws Exception
Specified by:
getObject in interface FactoryBean<Job>
Throws:
Exception

getObjectType

public Class<?> getObjectType()
Specified by:
getObjectType in interface FactoryBean<Job>

isSingleton

public boolean isSingleton()
Specified by:
isSingleton in interface FactoryBean<Job>

afterPropertiesSet

public void afterPropertiesSet()
                        throws Exception
Specified by:
afterPropertiesSet in interface InitializingBean
Throws:
Exception

processJob

protected void processJob(Job job)
                   throws Exception
Throws:
Exception

setConfiguration

public void setConfiguration(Configuration configuration)
Sets the Hadoop configuration to use.

Parameters:
configuration - The configuration to set.

setName

public void setName(String name)
Sets the job name.

Parameters:
name - The name to set.

setKey

public void setKey(String key)
Sets the job key class.

Parameters:
key - The keyClass to set.

setValue

public void setValue(String value)
Sets the job value class.

Parameters:
value - The valueClass to set.

setMapKey

public void setMapKey(String mapKey)
Sets the job map key class.

Parameters:
mapKey - The mapKeyClass to set.

setMapValue

public void setMapValue(String mapValue)
Sets the job map value class.

Parameters:
mapValue - The mapValueClass to set.

setMapper

public void setMapper(String mapper)
Sets the job mapper class.

Parameters:
mapper - The mapper to set.

setReducer

public void setReducer(String reducer)
Sets the job reducer class.

Parameters:
reducer - The reducer to set.

setCombiner

public void setCombiner(String combiner)
Sets the job combiner class.

Parameters:
combiner - The combiner to set.

setInputFormat

public void setInputFormat(String inputFormat)
Sets the job input format class.

Parameters:
inputFormat - The inputFormat to set.

setOutputFormat

public void setOutputFormat(String outputFormat)
Sets the job output format class.

Parameters:
outputFormat - The outputFormat to set.

setPartitioner

public void setPartitioner(String partitioner)
Sets the job partitioner class.

Parameters:
partitioner - The partitioner to set.

setSortComparator

public void setSortComparator(String sortComparator)
Sets the job sort comparator class.

Parameters:
sortComparator - The sortComparator to set.

setGroupingComparator

public void setGroupingComparator(String groupingComparator)
Sets the job grouping comparator class.

Parameters:
groupingComparator - The groupingComparator to set.

setWorkingDir

public void setWorkingDir(String workingDir)
Sets the job working directory.

Parameters:
workingDir - The workingDir to set.

setNumReduceTasks

public void setNumReduceTasks(Integer numReduceTasks)
Sets the number of reduce task for this job.

Parameters:
numReduceTasks - The numReduceTasks to set.

setJarByClass

public void setJarByClass(Class<?> jarClass)
Determines the job jar (available on the classpath) based on the given class.

Parameters:
jarClass - The jarClass to set.

setJar

public void setJar(Resource jar)
Sets the job jar (which might not be on the classpath).

Parameters:
jar - The jar to set.

setInputPath

public void setInputPath(String... inputPath)
Sets the job input path.

Parameters:
inputPath - job input path.

setOutputPath

public void setOutputPath(String outputPath)
Sets the job output path.

Parameters:
outputPath - The outputPath to set.

setCompressOutput

public void setCompressOutput(Boolean compressOutput)
Indicates whether the job output should be compressed or not.

Parameters:
compressOutput - The compressOutput to set.

setCodec

public void setCodec(String codecClass)
Sets the job codec class.

Parameters:
codecClass - The codecClass to set.

setValidatePaths

public void setValidatePaths(Boolean validatePaths)
Indicates whether the job input/output paths should be validated (default) before the job is submitted.

Parameters:
validatePaths - The validatePaths to set.

setProperties

public void setProperties(Properties properties)
The configuration properties to set for this job.

Parameters:
properties - The properties to set.

setLibs

public void setLibs(Resource... libJars)
Sets the jar files to include in the classpath. Note that a pattern can be used (e.g. mydir/*.jar), which the Spring container will automatically resolve.

Parameters:
libJars - The jar files to include in the classpath.

setFiles

public void setFiles(Resource... files)
Sets the files to be copied to the map reduce cluster. Note that a pattern can be used (e.g. mydir/*.txt), which the Spring container will automatically resolve.

Parameters:
files - The files to copy.

setArchives

public void setArchives(Resource... archives)
Sets the archives to be unarchive to the map reduce cluster. Note that a pattern can be used (e.g. mydir/*.zip), which the Spring container will automatically resolve.

Parameters:
archives - The archives to unarchive on the compute machines.

setUser

public void setUser(String user)
Sets the user impersonation (optional) for running this job. Should be used when running against a Hadoop Kerberos cluster.

Parameters:
user - user/group information