DB-Scheduler: A Persistent, Cluster-Friendly Scheduler for Java

1. Introduction

In this tutorial, we’ll take a look at db-scheduler, a simpler alternative to Quartz. We’ll see what it is, how to use it and what we can do with it.

2. What Is DB-Scheduler?

DB-Scheduler is a cluster-aware, persistent scheduling library. It’s designed to be simple to include in our application. Our only requirements are Java 17+ and a JDBC connection to a database with a single database table in which we store and manage our tasks.

Using db-scheduler, we can easily support:

• One-time tasks: tasks that are scheduled to run once at a specific point in time
• Simple recurring tasks: tasks that are set up to run on a statically defined regular schedule
• Dynamic recurring tasks: tasks that run on a regular schedule, but that we can schedule dynamically at runtime

The db-scheduler library will then handle everything for us. This includes ensuring that these jobs run correctly across a cluster of services without dropping them or running them multiple times.

3. Setting Up

Before we can use db-scheduler, we need to first set it up in our application. This includes adding it to our project and getting it running.

3.1. Dependencies

Before using db-scheduler, we need to include the latest version in our build, which is 16.12.0 at the time of writing.

If we’re using Maven, we can include this dependency in our pom.xml file:

<dependency>
    <groupId>com.github.kagkarlsson</groupId>
    <artifactId>db-scheduler</artifactId>
    <version>16.12.0</version>
</dependency>

This will bring in everything that we need to be able to use it in our application. The only required dependencies are JSpecify and SLF4J – everything else is self-contained.

3.2. Database Schema

In order to work, db-scheduler needs a single database table in the database we connect it to. DDL scripts are provided for most major RDBMS engines, though it might work on others as well. For example, to use with PostgreSQL, we’d use DDL like this:

CREATE TABLE scheduled_tasks (
  task_name TEXT NOT NULL,
  task_instance TEXT NOT NULL,
  task_data BYTEA,
  execution_time TIMESTAMP WITH TIME ZONE NOT NULL,
  picked BOOLEAN NOT NULL,
  picked_by TEXT,
  last_success TIMESTAMP WITH TIME ZONE,
  last_failure TIMESTAMP WITH TIME ZONE,
  consecutive_failures INT,
  last_heartbeat TIMESTAMP WITH TIME ZONE,
  version BIGINT NOT NULL,
  priority SMALLINT,
  PRIMARY KEY (task_name, task_instance)
);

CREATE INDEX execution_time_idx ON scheduled_tasks (execution_time);
CREATE INDEX last_heartbeat_idx ON scheduled_tasks (last_heartbeat);
CREATE INDEX priority_execution_time_idx on scheduled_tasks (priority desc, execution_time asc);

This creates our scheduled_tasks table with the expected structure, ready for db-scheduler to use.

3.3. Scheduler

Finally, before we can use db-scheduler, we need to actually set up a Scheduler instance. This is the central class that manages scheduling all of our jobs.

We create an instance of this by providing a DataSource pointing to the database containing our scheduled_tasks table, and a collection of all the tasks that we want to manage. We’ll see later how to create these:

Scheduler scheduler = Scheduler.create(dataSource)
  .startTasks(tasks)
  .registerShutdownHook()
  .build();

This comes with a reasonable set of default configuration values:

• Poll every 10 seconds
• Heartbeat every 5 minutes
• Consider a task dead if it misses 6 heartbeats
• Run jobs across 10 worker threads

And many more. We can configure all of these when we create our scheduler:

Scheduler scheduler = Scheduler.create(dataSource)
  .startTasks(tasks)
  .registerShutdownHook()
  .pollingInterval(Duration.ofSeconds(2))
  .heartbeatInterval(Duration.ofMinutes(2))
  .missedHeartbeatsLimit(10)
  .threads(5)
  .build();

It’s important that multiple instances in the same cluster use at least the same heartbeat settings. If not, db-scheduler might incorrectly consider some jobs to be dead.

Once we have a scheduler, we need to start it running:

scheduler.start();

At this point, the scheduler will now manage all of the tasks assigned to it. This includes ensuring they’re correctly scheduled and running them at the appropriate times.

4. Simple Recurring Tasks

The easiest tasks to work with are simple recurring tasks. For these, we need a unique name, a frequency at which the task runs, and the task itself:

RecurringTask<Void> task = Tasks.recurring("my-hourly-task", FixedDelay.ofHours(1))
  .execute((instance, context) -> {
    LOG.info("Executed!");
  });

If this is a new task that the scheduler has never seen before, it will be scheduled to run immediately. After that, the scheduler will run it according to the provided schedule. In this case, that means 1 hour after the previous iteration finished. We can also use Daily for running at a particular time every day, and CronSchedule for using a more complicated cron expression.

Our task itself is provided as an instance of VoidExecutionHandler<T>, which is itself a functional interface and can be implemented by a lambda if we wish. The only method of this interface takes two parameters:

• TaskInstance<T> instance – Details about this instance of the task. Useful if we’ve got multiple schedules using the same task implementation.
• ExecutionContext context – Details about the executor itself. Useful if we want to access the scheduler itself from within our task.

Note that we have VoidExecutionHandler and RecurringTask<Void> here. This is because our example task is stateless. We can also support stateful tasks, which have an initial state that can be updated as a result of the task executing. However, these are out of the scope of this article.

5. One-Time Tasks

One-time tasks work a bit differently. In this case, we need to distinguish between the task itself and a descriptor that identifies the task:

TaskDescriptor<String> taskDescriptor = TaskDescriptor.of("my-onetime-task", String.class);

Task<String> task = Tasks.oneTime(taskDescriptor)
  .execute((inst, ctx) -> {
    LOG.info("Executed! Custom data {}, Instance {}", inst.getData(), inst.getId());
  });

This task definition accepts some data that we provide when we schedule the task, and can act on that data as appropriate. In this case, the data is a String, but it can be any type that db-scheduler is able to serialize. By default, this uses Java Serialization but db-scheduler also supports other mechanisms, such as Jackson and Gson.

We then need to register the task with our scheduler. However, we do this by passing the task to the Scheduler.create() method so that db-scheduler knows about it but doesn’t immediately start running it:

Scheduler scheduler = Scheduler.create(dataSource, task)
  .build();

At this point, we can run our task at any time using our scheduler and task descriptor:

scheduler.schedule(taskDescriptor.instance(UUID.randomUUID().toString())
  .data("Hello")
  .scheduledTo(Instant.now().plusSeconds(5))
);

Here we have to provide a unique instance ID for our task, and indicate when the task should run. We’ve also provided some data for our task to act on. The scheduler will then ensure that this task runs correctly at the desired time.

6. Dynamic Recurring Tasks

Dynamic recurring tasks are tasks that are registered dynamically at runtime, similar to our one-time tasks. However, once registered, they’ll continue to run on a schedule similar to our simple recurring tasks.

We create and start these exactly the same as for our one-time task, only using Tasks.recurring() instead:

TaskDescriptor<String> taskDescriptor = TaskDescriptor.of("my-dynamic-recurring-task", String.class);

Task<String> task = Tasks.recurring(taskDescriptor, new CronSchedule("*/5 * * * * ?", ZoneId.of("UTC")))
  .execute((inst, ctx) -> {
    LOG.info("Executed! Custom data {}, Instance {}", inst.getData(), inst.getId());
  });

scheduler.schedule(taskDescriptor.instance(UUID.randomUUID().toString())
  .data("Hello")
  .scheduledTo(Instant.now().plusSeconds(15))
);

This will now start our task at the given time and then, once started, it’ll run at the given schedule. In this case, it first runs 15 seconds after the current time, and then every 5 seconds after that.

If we wish, we can schedule as many different instances of these as we need, and they’ll all run on their own schedule without interfering with each other.

7. Use With Spring Boot

If we’re using Spring Boot, db-scheduler provides a starter dependency that we can use:

<dependency>
    <groupId>com.github.kagkarlsson</groupId>
    <artifactId>db-scheduler-spring-boot-4-starter</artifactId>
    <version>16.12.0</version>
</dependency>

If we’re using Spring Boot 4.x then we need to use db-scheduler-spring-boot-4-starter. If we’re still on Spring Boot 3.x then instead it’s db-scheduler-spring-boot-starter.

This will automatically create and start our Scheduler instance as a Spring bean. On doing this, it’ll also automatically discover any tasks that have been created as Spring beans and register them with this scheduler. This means that the only thing we need to do is create the tasks themselves.

If we want to configure the Scheduler in any way, we can do this using properties in our standard way:

db-scheduler.enabled=true
db-scheduler.polling-interval=5s
db-scheduler.heartbeat-interval=2m
db-scheduler.missed-heartbeats-limit=10
db-scheduler.threads=5

Everything else about using db-scheduler, including how we create and schedule tasks, is exactly the same as before. The starter just makes it simpler to get started in the first place.

8. Conclusion

In this article, we took a very quick look at db-scheduler, including how to schedule recurring and one-time tasks. There’s a lot more that we can do with this. Next time we need to manage scheduled jobs for your applications, why not give it a try?

As always, all the code from this article is available over on GitHub.