Access Job Parameters From ItemReader in Spring Batch

1. Overview

Spring Batch is a powerful framework for batch processing in Java, thus making it a popular choice for data processing activities and scheduled job runs. Depending on the business logic complexity, a job can rely on different configuration values and dynamic parameters.

In this article, we’ll explore how to work with JobParameters and how to access them from essential batch components.

2. Demo Setup

We’ll develop a Spring Batch for a pharmacy service. The main business task is to find medications that expire soon, calculate new prices based on sales, and notify consumers about meds that are about to expire. Additionally, we’ll read from the in-memory H2 database and write all processing details to logs to simplify implementation.

2.1. Dependencies

To start with the demo application, we need to add Spring Batch and H2 dependencies:

<dependency>
    <groupId>com.h2database</groupId>
    <artifactId>h2</artifactId>
    <version>2.2.224</version>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-batch</artifactId>
    <version>3.2.0</version>
</dependency>

We can find the latest H2 and Spring Batch versions in the Maven Central repository.

2.2. Prepare Test Data

Let’s start by defining the schema in schema-all.sql:

DROP TABLE medicine IF EXISTS;

CREATE TABLE medicine  (
    med_id VARCHAR(36) PRIMARY KEY,
    name VARCHAR(30),
    type VARCHAR(30),
    expiration_date TIMESTAMP,
    original_price DECIMAL,
    sale_price DECIMAL
);

Initial test data is provided in data.sql:

INSERT INTO medicine VALUES ('ec278dd3-87b9-4ad1-858f-dfe5bc34bdb5', 'Lidocaine', 'ANESTHETICS', DATEADD('DAY', 120, CURRENT_DATE), 10, null);
INSERT INTO medicine VALUES ('9d39321d-34f3-4eb7-bb9a-a69734e0e372', 'Flucloxacillin', 'ANTIBACTERIALS', DATEADD('DAY', 40, CURRENT_DATE), 20, null);
INSERT INTO medicine VALUES ('87f4ff13-de40-4c7f-95db-627f309394dd', 'Amoxicillin', 'ANTIBACTERIALS', DATEADD('DAY', 70, CURRENT_DATE), 30, null);
INSERT INTO medicine VALUES ('acd99d6a-27be-4c89-babe-0edf4dca22cb', 'Prozac', 'ANTIDEPRESSANTS', DATEADD('DAY', 30, CURRENT_DATE), 40, null);

Spring Boot runs these files as part of the application startup and we’ll use these test data in our test executions.

2.3. Medicine Domain Class

For our service, we’ll need a simple Medicine entity class:

@AllArgsConstructor
@Data
public class Medicine {
    private UUID id;
    private String name;
    private MedicineCategory type;
    private Timestamp expirationDate;
    private Double originalPrice;
    private Double salePrice;
}

ItemReader uses the expirationDate field to calculate if the medication expires soon. The salePrice field will be updated by ItemProcessor when the medication is close to the expiration date.

2.4. Application Properties

The application needs multiple properties in the src/main/resources/application.properties file:

spring.batch.job.enabled=false
batch.medicine.cron=0 */1 * * * *
batch.medicine.alert_type=LOGS
batch.medicine.expiration.default.days=60
batch.medicine.start.sale.default.days=45
batch.medicine.sale=0.1

As we’ll configure only one job, spring.batch.job.enabled should be set to false to disable the initial job execution. By default, Spring runs the job after the context startup with empty parameters:

[main] INFO  o.s.b.a.b.JobLauncherApplicationRunner - Running default command line with: []

Other properties are needed for InputReader, InputProcessor, and InpurWriter to perform business logic.

3. Job Parameters

Spring Batch includes a JobParameters class designed to store runtime parameters for a particular job run. This functionality proves beneficial in various situations. For instance, it allows the passing of dynamic variables generated during a specific run. Moreover, it makes it possible to create a controller that can initiate a job based on parameters provided by the client.

In our scenario, we’ll utilize this class to hold application parameters and dynamics runtime parameters.

3.1. StepScope and JobScope

In addition to the well-known bean scopes in regular Spring, Spring Batch introduces two additional scopes: StepScope and JobScope. With these scopes, it becomes possible to create unique beans for each step or job in a workflow. Spring ensures that the resources associated with a particular step/job are isolated and managed independently throughout its lifecycle.

Having this feature, we can easily control contexts and share all the needed properties across read, process, and write parts for specific runs. To be able to inject job parameters we need to annotate depending beans with @StepScope or @JobScope.

3.2. Populate Job Parameters in Scheduled Execution

Let’s define the MedExpirationBatchRunner class that will start our job by cron expression (every 1 minute in our case). We should annotate the class with @EnableScheduling and define the appropriate @Scheduled entry method:

@Component
@EnableScheduling
public class MedExpirationBatchRunner {
    ...
    @Scheduled(cron = "${batch.medicine.cron}", zone = "GMT")
    public void runJob() {
        ZonedDateTime now = ZonedDateTime.now(ZoneOffset.UTC);
        launchJob(now);
    }
}

As we want to launch the job manually, we should use the JobLaucher class and provide a populated JobParameter in JobLauncher#run() method. In our example, we’ve provided values from application.properties as well as two run-specific parameters (date when the job got triggered and trace id):

public void launchJob(ZonedDateTime triggerZonedDateTime) {
    try {
        JobParameters jobParameters = new JobParametersBuilder()
          .addString(BatchConstants.TRIGGERED_DATE_TIME, triggerZonedDateTime.toString())
          .addString(BatchConstants.ALERT_TYPE, alertType)
          .addLong(BatchConstants.DEFAULT_EXPIRATION, defaultExpiration)
          .addLong(BatchConstants.SALE_STARTS_DAYS, saleStartDays)
          .addDouble(BatchConstants.MEDICINE_SALE, medicineSale)
          .addString(BatchConstants.TRACE_ID, UUID.randomUUID().toString())
          .toJobParameters();

        jobLauncher.run(medExpirationJob, jobParameters);
    } catch (Exception e) {
        log.error("Failed to run", e);
    }
}

After configuring parameters, we have several options for how to use these values in code.

3.3. Read Job Parameters in Bean Definition

Using SpEL we can access job parameters from a bean definition in our configuration class. Spring combines all parameters to a regular String to Object map:

@Bean
@StepScope
public MedicineProcessor medicineProcessor(@Value("#{jobParameters}") Map<String, Object> jobParameters) {
    ...
}

Inside the method, we’ll use jobParameters to initiate the proper fields of MedicineProcessor.

3.4. Read Job Parameters in Service Directly

Another option is to use setter injection in the ItemReader itself. We can fetch the exact parameter value just like from any other map via SpEL expression:

@Setter
public class ExpiresSoonMedicineReader extends AbstractItemCountingItemStreamItemReader<Medicine> {

    @Value("#{jobParameters['DEFAULT_EXPIRATION']}")
    private long defaultExpiration;
}

We just need to ensure that the key used in SpEL is the same as the key used during parameters initialization.

3.5. Read Job Parameters via Before Step

Spring Batch provides a StepExecutionListener interface that allows us to listen for step execution phases: before the step starts and once the step is completed. We can utilize this feature, access properties before the step is started, and perform any custom logic. The easiest way is just to use @BeforeStep annotation which corresponds to beforeStep() method from StepExecutionListener:

@BeforeStep
public void beforeStep(StepExecution stepExecution) {
    JobParameters parameters = stepExecution.getJobExecution()
      .getJobParameters();
    ...
    log.info("Before step params: {}", parameters);
}

4. Job Configuration

Let’s combine all the parts to see the whole picture.

There are two properties, that are required for the reader, processor, and writer: BatchConstants.TRIGGERED_DATE_TIME and BatchConstants.TRACE_ID. 

We’ll use the same extraction logic for common parameters from all step bean definitions:

private void enrichWithJobParameters(Map<String, Object> jobParameters, ContainsJobParameters container) {
    if (jobParameters.get(BatchConstants.TRIGGERED_DATE_TIME) != null) {
        container.setTriggeredDateTime(ZonedDateTime.parse(jobParameters.get(BatchConstants.TRIGGERED_DATE_TIME)
          .toString()));
    }
    if (jobParameters.get(BatchConstants.TRACE_ID) != null) {
        container.setTraceId(jobParameters.get(BatchConstants.TRACE_ID).toString());
    }
}

Altogether other parameters are component-specific and don’t have common logic.

4.1. Configuring ItemReader

At first, we want to configure ExpiresSoonMedicineReader and enrich common parameters:

@Bean
@StepScope
public ExpiresSoonMedicineReader expiresSoonMedicineReader(JdbcTemplate jdbcTemplate, @Value("#{jobParameters}") Map<String, Object> jobParameters) {

    ExpiresSoonMedicineReader medicineReader = new ExpiresSoonMedicineReader(jdbcTemplate);
    enrichWithJobParameters(jobParameters, medicineReader);
    return medicineReader;
}

Let’s take a closer look at the exact reader implementation. TriggeredDateTime and traceId parameters are injected directly during bean construction, defaultExpiration parameter is injected by Spring via setter. For demonstration, we have used all of them in doOpen() method:

public class ExpiresSoonMedicineReader extends AbstractItemCountingItemStreamItemReader<Medicine> implements ContainsJobParameters {

    private ZonedDateTime triggeredDateTime;
    private String traceId;
    @Value("#{jobParameters['DEFAULT_EXPIRATION']}")
    private long defaultExpiration;

    private List<Medicine> expiringMedicineList;

    ...

    @Override
    protected void doOpen() {
        expiringMedicineList = jdbcTemplate.query(FIND_EXPIRING_SOON_MEDICINE, ps -> ps.setLong(1, defaultExpiration), (rs, row) -> getMedicine(rs));

        log.info("Trace = {}. Found {} meds that expires soon", traceId, expiringMedicineList.size());
        if (!expiringMedicineList.isEmpty()) {
            setMaxItemCount(expiringMedicineList.size());
        }
    }

    @PostConstruct
    public void init() {
        setName(ClassUtils.getShortName(getClass()));
    }

}

ItemReader should not be marked as @Component. Also, we need to call setName() method to set the required reader name.

4.2. Configuring ItemProcessor and ItemWriter

ItemProcessor and ItemWriter follow the same approaches as ItemReader. So they don’t require any specific configuration to access parameters. The bean definition logic initializes common parameters through the enrichWithJobParameters() method. Other parameters, that are used by a single class and do not need to be populated in all components, are enriched by Spring through setter injection in the corresponding classes.

We should mark all properties-dependent beans with @StepScope annotation. Otherwise, Spring will create beans only once at context startup and will not have parameters’ values to inject.

4.3. Configuring Complete Flow

We don’t need to take any specific action to configure the job with parameters. Therefore we just need to combine all the beans:

@Bean
public Job medExpirationJob(JobRepository jobRepository,
    PlatformTransactionManager transactionManager,
    MedicineWriter medicineWriter,
    MedicineProcessor medicineProcessor,
    ExpiresSoonMedicineReader expiresSoonMedicineReader) {
    Step notifyAboutExpiringMedicine = new StepBuilder("notifyAboutExpiringMedicine", jobRepository).<Medicine, Medicine>chunk(10)
      .reader(expiresSoonMedicineReader)
      .processor(medicineProcessor)
      .writer(medicineWriter)
      .faultTolerant()
      .transactionManager(transactionManager)
      .build();

    return new JobBuilder("medExpirationJob", jobRepository)
      .incrementer(new RunIdIncrementer())
      .start(notifyAboutExpiringMedicine)
      .build();
}

5. Running the Application

Let’s run a complete example and see how the application uses all parameters. We need to start the Spring Boot application from the SpringBatchExpireMedicationApplication class.

As soon as the scheduled method executes, Spring logs all parameters:

INFO  o.s.b.c.l.support.SimpleJobLauncher - Job: [SimpleJob: [name=medExpirationJob]] launched with the following parameters: [{'SALE_STARTS_DAYS':'{value=45, type=class java.lang.Long, identifying=true}','MEDICINE_SALE':'{value=0.1, type=class java.lang.Double, identifying=true}','TRACE_ID':'{value=e35a26a4-4d56-4dfe-bf36-c1e5f20940a5, type=class java.lang.String, identifying=true}','ALERT_TYPE':'{value=LOGS, type=class java.lang.String, identifying=true}','TRIGGERED_DATE_TIME':'{value=2023-12-06T22:36:00.011436600Z, type=class java.lang.String, identifying=true}','DEFAULT_EXPIRATION':'{value=60, type=class java.lang.Long, identifying=true}'}]

Firstly, ItemReader writes info about meds that have been found based on the DEFAULT_EXPIRATION parameter:

INFO  c.b.b.job.ExpiresSoonMedicineReader - Trace = e35a26a4-4d56-4dfe-bf36-c1e5f20940a5. Found 2 meds that expires soon

Secondly, ItemProcessor uses SALE_STARTS_DAYS and MEDICINE_SALE parameters to calculate new prices:

INFO  c.b.b.job.MedicineProcessor - Trace = e35a26a4-4d56-4dfe-bf36-c1e5f20940a5, calculated new sale price 18.0 for medicine 9d39321d-34f3-4eb7-bb9a-a69734e0e372
INFO  c.b.b.job.MedicineProcessor - Trace = e35a26a4-4d56-4dfe-bf36-c1e5f20940a5, calculated new sale price 36.0 for medicine acd99d6a-27be-4c89-babe-0edf4dca22cb

Lastly, ItemWriter writes updated medications to logs within the same trace:

INFO  c.b.b.job.MedicineWriter - Trace = e35a26a4-4d56-4dfe-bf36-c1e5f20940a5. This medicine is expiring Medicine(id=9d39321d-34f3-4eb7-bb9a-a69734e0e372, name=Flucloxacillin, type=ANTIBACTERIALS, expirationDate=2024-01-16 00:00:00.0, originalPrice=20.0, salePrice=18.0)
INFO  c.b.b.job.MedicineWriter - Trace = e35a26a4-4d56-4dfe-bf36-c1e5f20940a5. This medicine is expiring Medicine(id=acd99d6a-27be-4c89-babe-0edf4dca22cb, name=Prozac, type=ANTIDEPRESSANTS, expirationDate=2024-01-06 00:00:00.0, originalPrice=40.0, salePrice=36.0)
INFO  c.b.b.job.MedicineWriter - Finishing job started at 2023-12-07T11:58:00.014430400Z

6. Conclusion

In this article, we’ve learned how to work with Job Parameters in Spring Batch. ItemReader, ItemProcessor, and ItemWriter can be manually enriched with parameters during bean initialization or might be enriched by Spring via @BeforeStep or setter injection.

As always, the complete examples are available over on GitHub.