Restart a Job on Failure and Continue in Spring Batch

1. Introduction

Spring Batch provides robust mechanisms for restarting failed jobs. These mechanisms allow jobs to resume processing from the point of failure. This capability is essential for handling large-scale data processing tasks efficiently.

Spring Batch’s built-in JobRepository persists job execution state. This allows for default restartability of jobs. As a result, a failed job can resume exactly where it left off. This ensures that no duplicate processing or data loss occurs.

In this tutorial, we’ll look at how to configure and restart a failed Spring Batch job effectively.

2. Maven Dependencies

Let’s start by importing the spring-boot-starter-batch, spring-boot-starter-data-jpa, and h2 dependencies to our pom.xml:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-batch</artifactId>
    <version>3.3.2</version>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-jpa</artifactId>
    <version>3.3.2</version>
</dependency>
	  
<dependency>
    <groupId>com.h2database</groupId>
    <artifactId>h2</artifactId>
    <version>2.2.224</version>
</dependency>

We need the file-based H2 database to enable job restartability by persisting job execution state across application runs.

3. Defining a Simple Spring Batch Job

In this section, we’ll explore a Spring Batch job configuration that demonstrates a simple batch processing workflow. We’ll define a job with one step: processing a CSV file.

In Spring Boot 3, we should avoid using @EnableBatchProcessing as it disables Spring Boot’s helpful auto-configuration (like creation of Spring Batch tables).

3.1. Configuration

Let’s create the BatchConfig class, which sets up a job named simpleJob:

@Configuration
public class BatchConfig {

    @Bean
    public Job simpleJob(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
        return new JobBuilder("simpleJob", jobRepository)
          .start(step1(jobRepository, transactionManager))
          .build();
    }
}

The simpleJob bean defines the batch job using JobBuilder. The job consists of one step: step1, which reads, processes, and writes a CSV file:

@Bean
public Step step1(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
    return new StepBuilder("step1", jobRepository)
      .<String, String>chunk(2, transactionManager)
      .reader(flatFileItemReader())
      .processor(itemProcessor())
      .writer(itemWriter())
      .build();
}

Step 1 defines a chunk-based step using the StepBuilder. This step processes data in batches of two items at a time, reading strings from a FlatFileItemReader, transforming them with an ItemProcessor, and writing the results using an ItemWriter – all managed within a transaction controlled by the PlatformTransactionManager to ensure data integrity.

The JobRepository persists the step’s execution state, including the FlatFileItemReader’s position, enabling restartability from the last uncommitted chunk if the job fails.

3.2. Item Reader

Let’s define the flatFileItemReader bean that provides the input data:

@Bean
@StepScope
public FlatFileItemReader<String> flatFileItemReader() {
    return new FlatFileItemReaderBuilder<String>()
      .name("itemReader")
      .resource(new ClassPathResource("data.csv"))
      .lineMapper(new PassThroughLineMapper())
      .saveState(true)
      .build();
}

The code defines a FlatFileItemReader<String> bean named flatFileItemReader with @StepScope to ensure a new instance is created for each step execution, enabling proper state management and restartability. It reads strings from a data.csv file located in the classpath, maps each line to a string using a PassThroughLineMapper.

Also, it has saveState(true) to persist its read position in the ExecutionContext. This allows the reader to resume from the last unprocessed line in case of a job failure, leveraging the JobRepository for state persistence.

For a job to be truly restartable, the ItemReader must persist its state in Spring Batch’s execution context. Readers like FlatFileItemReader automatically save critical progress information (such as line numbers or record counts) between chunks.

3.3. Item Processor

Let’s declare the itemProcessor bean that transforms the input data:

@Bean
public RestartItemProcessor itemProcessor() {
    return new RestartItemProcessor();
}

static class RestartItemProcessor implements ItemProcessor<String, String> {
    private boolean failOnItem3 = true;

    public void setFailOnItem3(boolean failOnItem3) {
        this.failOnItem3 = failOnItem3;
    }

    @Override
    public String process(String item) throws Exception {
        System.out.println("Processing: " + item + " (failOnItem3=" + failOnItem3 + ")");
        if (failOnItem3 && item.equals("Item3")) {
            throw new RuntimeException("Simulated failure on Item3");
        }
        return "PROCESSED " + item;
    }
}

It processes each item by prefixing it with PROCESSED and simulates a failure on Item3.

3.4. Item Writer

Now, let’s create the itemWriter bean that outputs the processed data:

@Bean
public ItemWriter<String> itemWriter() {
    return items -> {
      System.out.println("Writing items:");
      for (String item : items) {
          System.out.println("- " + item);
      }
    };
}

This prints processed items to the console. Now, our application is ready.

4. Restarting a Failed Spring Batch Job

Spring Batch jobs are designed to be restartable by default, allowing them to seamlessly resume from the point of failure. Consequently, no additional configuration is required to enable this functionality.

However, for this to work effectively, the job state must be persisted in a JobRepository. Furthermore, the JobRepository must be backed by a database to ensure reliable storage and retrieval of the job’s execution state.

The following subsections describe how to simulate a job failure and restart it.

4.1. Simulating a Job Failure

To simulate a job failure, the ItemProcessor is configured to throw a RuntimeException when processing Item3. When the failure occurs on Item3, the JobRepository stores this state, marking the job as FAILED.

Running the application with mvn spring-boot:run produces:

Starting new job execution...
Processing: Item1
Processing: Item2
Writing items:
- PROCESSED Item1
- PROCESSED Item2
Processing: Item3
[Exception: Simulated failure on Item3]
Job started with status: FAILED

This output confirms that Item1 and Item2 are processed and written, but the failure on Item3 halts the job, with the state persisted for a subsequent restart.

4.2. Restarting the Job

To restart the failed job, we use the CommandLineRunner to detect the failed job instance using the JobExplorer and the fixed JobParameters:

@Bean
CommandLineRunner run(JobLauncher jobLauncher, Job job, JobExplorer jobExplorer,
    JobOperator jobOperator, BatchConfig.RestartItemProcessor itemProcessor) {
    return args -> {
      JobParameters jobParameters = new JobParametersBuilder()
        .addString("jobId", "test-job-" + System.currentTimeMillis())
        .toJobParameters();

      List<JobInstance> instances = jobExplorer.getJobInstances("simpleJob", 0, 1);
      if (!instances.isEmpty()) {
          JobInstance lastInstance = instances.get(0);
          List<JobExecution> executions = jobExplorer.getJobExecutions(lastInstance);
          if (!executions.isEmpty()) {
              JobExecution lastExecution = executions.get(0);
              if (lastExecution.getStatus() == BatchStatus.FAILED) {
                  itemProcessor.setFailOnItem3(false);

                  JobExecution restartedExecution = jobLauncher.run(job, jobParameters);

                  // final Long restartId = jobOperator.restart(lastExecution.getId());
                  // final JobExecution restartedExecution = jobExplorer.getJobExecution(restartedExecution);

                  // ...
              }
          }
      }

    };
}

The code checks the job repository via JobExplorer for any existing instances of simpleJob.

It detects a failed execution with a status of FAILED. It uses JobLauncher.run() or JobOperator.restart() to resume that specific job. The job resumes from its last persisted state. This ensures previously processed items aren’t reprocessed.

To ignore the failure on Item3 during the restart, we set itemProcessor.setFailOnItem3(false) before launching the restarted job, allowing the RestartItemProcessor to process Item3 without throwing an exception.

Now, we run the application again.

Let’s check the output:

Restarting failed job execution with ID: [execution_id]
Processing: Item3
Processing: Item4
Writing items:
- PROCESSED Item3
- PROCESSED Item4
Processing: Item5
Writing items:
- PROCESSED Item5
Restarted job status: COMPLETED

The Spring Batch job had failed on Item3. It successfully restarted from the failure point. Then, the job processed Item3 to Item5 in chunks, wrote the results, and completed with a COMPLETED status.

4.3. Testing the Job Restart

Let’s add a unit test to verify Spring Batch’s job restart functionality instead of using a CommandLineRunner:

@Test
public void givenItems_whenFailed_thenRestartFromFailure() throws Exception {
    // Given
    createTestFile("Item1\nItem2\nItem3\nItem4");

    JobParameters jobParameters = new JobParametersBuilder()
      .addLong("time", System.currentTimeMillis())
      .toJobParameters();

    // When
    JobExecution firstExecution = jobLauncherTestUtils.launchJob(jobParameters);
    assertEquals(BatchStatus.FAILED, firstExecution.getStatus());

    Long executionId = firstExecution.getId();

    itemProcessor.setFailOnItem3(false);

    // Then
    JobExecution restartedExecution = jobLauncherTestUtils.launchJob(jobParameters);

    assertEquals(BatchStatus.COMPLETED, restartedExecution.getStatus());

    assertEquals(
      firstExecution.getJobInstance().getInstanceId(),
      restartedExecution.getJobInstance().getInstanceId()
    );
}

The test method first executes a job that intentionally fails when processing Item3 (asserting the expected FAILED status). Then, it modifies the processor behavior to no longer fail on that item. Finally, it restarts the job with the same parameters to confirm it can successfully complete from the point of failure.

The test validates three key aspects. First, it confirms that the initial failure occurs as designed, testing the fault simulation. Next, it ensures the restarted job continues processing from the point of failure, verifying restart logic. Finally, it checks that both executions belong to the same job instance, validating instance tracking.

4.4. Preventing the Job Restart

We can prevent to restart the job using the preventRestart() method:

@Bean
public Job simpleJob(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
    return new JobBuilder("simpleJob", jobRepository)
      .start(step1(jobRepository, transactionManager))
      .preventRestart()
      .build();
}

Adding .preventRestart() to the JobBuilder configures the job to always start from the beginning (e.g., Item1) rather than resuming from the point of failure (e.g., Item3) when relaunched. This overrides Spring Batch’s default behavior of persisting the job state in the JobRepository for restart purposes.

5. Conclusion

Spring Batch’s default restartability enables robust recovery from job failures, ensuring that a failed job can resume from the point of failure without reprocessing completed items or losing data.

In this article, we created a simple job that demonstrated this restart capability. We configured a job to process items in chunks, simulating a failure at Item3. Upon restarting, the job resumed from Item3, processed through Item5, and completed successfully with a COMPLETED status.

We also explored how to override this behavior using preventRestart() on the JobBuilder. This forces the job to start from the beginning, like Item1, instead of resuming from the failure point.