The most likely thing you will want to customize is the location of the database. Spring Batch Admin ships with an embedded HSQLDB database, which is initialized on start up.
- To change the database type add a file to the application classpath called batch-[type].properties, where [type] is the database type you want to use (e.g. mysql, oracle, db2). Copy the contents of the batch-hsql.properties from the Manager jar and change the values to suit your environment. Then launch the application with a system property -DENVIRONMENT=[type].
- To stop the database from being wiped and re-created on start up just set batch.data.source.init=false (in the properties file or as a System property).
N.B. The use of the environment variable to switch on the database type is only a suggestion. You can change that and other things by overriding and adding configuration fragments to the Spring application context. See below for details.
Overriding Components from Spring Batch Admin
The system tries to provide some useful defaults for things like transaction manager, job repository, job registry etc. Most of these live in the manager jar in a special place: META-INF/spring/batch/bootstrap. If you want to override them, just add your own versions of the same bean definitions to a Spring XML config file in META-INF/spring/batch/override (these are guaranteed to load after the bootstrap files, so they can override default definitions). You could use this to override the data source definition as an alternative to the environment settings described above.
Important Bootstrap Components
The important bootstrap components are listed below along with some common scenarios for when you might want to override or replace them:
JDBC Data Source
- Bean ID: dataSource
- Default: DBCP BasicDataSource with placeholders for common properties.
- Override Scenarios: Change to another implememtation, or add additional placeholders.
Spring Transaction Manager
- Bean ID: transactionManager
- Default: DataSourceTransactionManager injected with the dataSource
- Override Scenarios: Change to another implementation. For example if you are using Hibernate in your jobs, you will need to provide a HibernateTransactionManager.
- Bean ID: jobLauncher
- Default: SimpleJobLauncher injected with the jobRepository and transactionManager, and also a thread pool task executor with bean id poolTaskExector, so that launches happen in a background thread by default.
- Override Scenarios: Should be fine as it is for most use cases. To switch to synchronous launching change the task executor (see below).
Job Launcher Task Executor
- Bean ID: jobLauncherTaskExecutor
- Default: a task executor with pool-size=6 and rejection policy ABORT.
- Description: Injected into the jobLauncher to control the number of cocurrent jobs executing in-process. The effect of the defaults is that at most 6 jobs will run concurrently when launched locally from this application, and any submitted in excess of that will fail.
- Override Scenarios:
- Change to a synchronous task executor for integration testing (see the sample unit tests for an example), e.g.
<bean id="jobLauncherTaskExecutor" class="org.springframework.core.task.SyncTaskExecutor"/>
- Change the pool parameters. Note that these settings can be changed at runtime using JMX, if the task executor is exposed explictly.
Throttled Task Executor for Jobs and Steps
- Bean ID: throttledTaskExecutor
- Default: a task executor with pool-size=600 and rejection policy CALLER_RUNS and infinite throttle limit.
- Description: Not used anywhere in the manager, but provided as a convenience for applications that provide their own jobs and steps with concurrent behaviour.
- Override Scenarios: The sample contains one job with a concurrent step ("job2"), and it works by creating a local taskExecutor bean inheriting from the root definition, and adding a throttle limit:
This task executor is then injected into a step in the sample which has the effect of limiting the concurrency of the step to 100, but within overall limits set globally by the bootstrap context, and without discarding or failing any tasks that are submitted. The throttle limit is set to 100 so that if this job is the only one running it will hardly be affected by the throttle, but if it is contending with other jobs, then they will all eventually run in the thread pool provided by the bootstrap (with 600 threads by default).
<bean id="taskExecutor" parent="throttledTaskExecutor">
<property name="throttleLimit" value="100"/>
The throttled executor itself delegates to a pool which can be modified independently by overriding its bean definition poolTaskExecutor, e.g.
<task:executor id="poolTaskExecutor" pool-size="200" rejection-policy="CALLER_RUNS"/>
changes the pool size to 200 (from 600), limiting all jobs that use it to at most 200 threads.
Your mileage may vary with these thread pool settings depending on the workload that is executed. It is worth experimenting with the parameters to see if you can affect the overall throughput or execution times of your multi-threaded jobs.
- Bean ID: jobRepository
- Default: a JDBC version of SimpleJobRepository.
- Override Scenarios: Not common to override. The most likely scenario is changing the table prefix or large string column sizes (e.g. for a 2-byte character encoding the maxVarCharLength property would be set to 1/2 the length of the long columns in the database).
- Bean ID: jobLoader
- Default: a AutomaticJobRegistrar with path locations set to classpath*:/META-INF/spring/batch/jobs/*.xml.
- Override Scenarios:
- Changing the path locations maybe (trivial but not that useful).
- Change the inheritance strategy for post processors. By default the AOP configuration and the PropertyPLaceHolderConfigurer instances from the root context are copied into the child context created from the Job configuration files. You can change this behaviour just by providing your own versions in addition in the child context, or to make global changes you can override the jobLoader with one that copies different post processors down into the child, or one that does something completely different.
- Override it to a dummy bean for the purposes of an integration test, so that it doesn't try to create multiple instances of the Job that you want to test. The sample application has an example. In JobExecutionTests-context.xml we want to load the Jobs explicitly into the test context, and prevent them from being loaded again by the job loader, so we do this:
<!-- prevent loading of other jobs by overriding the loader in the main bootstrap context -->
<bean id="jobLoader" class="java.lang.String"/>
- Bean ID: jobService
- Default: a JDBC version of SimpleJobService.
- Override Scenarios: Not common to override. The job service is configured through a factory bean a bit like the JobRepository, and it has the same basic properties (e.g. you can change the table prefix or long column length).