1. Overview
Flyway migrations don’t always go according to plan. In this tutorial, we’ll explore the options we have for recovering from a failed migration.
2. Setup
Let’s start with a basic Flyway configured Spring Boot project. It has the flyway-core, spring-boot-starter-jdbc, and flyway-maven-plugin dependencies.
For more configuration details please refer to our article that introduces Flyway.
2.1. Configuration
First, let’s add two different profiles. This will enable us to easily run migrations against different database engines:
<profile>
<id>h2</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<dependencies>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
</dependency>
</dependencies>
</profile>
<profile>
<id>postgre</id>
<dependencies>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
</dependency>
</dependencies>
</profile>
Let’s also add the Flyway database configuration files for each of these profiles.
First, we create the application-h2.properties:
flyway.url=jdbc:h2:file:./testdb;DB_CLOSE_ON_EXIT=FALSE;AUTO_RECONNECT=TRUE;MODE=MySQL;DATABASE_TO_UPPER=false;
flyway.user=testuser
flyway.password=password
And after that, let’s create the PostgreSQL application-postgre.properties:
flyway.url=jdbc:postgresql://127.0.0.1:5431/testdb
flyway.user=testuser
flyway.password=password
Note: We can either adjust the PostgreSQL configuration to match an already existing database, or we can use the docker-compose file in the code sample.
2.2. Migrations
Let’s add our first migration file, V1_0__add_table.sql:
create table table_one (
id numeric primary key
);
Now let’s add a second migration file that contains an error, V1_1__add_table.sql:
create table <span style="color: #ff0000">table_one</span> (
id numeric primary key
);
We’ve made a mistake on purpose by using the same table name. This should lead to a Flyway migration error.
3. Run the Migrations
Now, let’s run the application and try to apply the migrations.
First for the default h2 profile:
mvn spring-boot:run
Then for the postgre profile:
mvn spring-boot:run -Ppostgre
As expected, the first migration was successful, while the second failed:
Migration V1_1__add_table.sql failed
...
Message : Table "TABLE_ONE" already exists; SQL statement:
3.1. Checking the State
Before moving on to repair the database, let’s inspect the Flyway migration state by running:
mvn flyway:info -Ph2
This returns, as expected:
+-----------+---------+-------------+------+---------------------+---------+
| Category | Version | Description | Type | Installed On | State |
+-----------+---------+-------------+------+---------------------+---------+
| Versioned | 1.0 | add table | SQL | 2020-07-17 12:57:35 | Success |
| Versioned | 1.1 | add table | SQL | 2020-07-17 12:57:35 | <span style="color: #ff0000">Failed</span> |
+-----------+---------+-------------+------+---------------------+---------+
But when we check the state for PostgreSQL with:
mvn flyway:info -Ppostgre
We notice the state of the second migration is Pending and not Failed:
+-----------+---------+-------------+------+---------------------+---------+
| Category | Version | Description | Type | Installed On | State |
+-----------+---------+-------------+------+---------------------+---------+
| Versioned | 1.0 | add table | SQL | 2020-07-17 12:57:48 | Success |
| Versioned | 1.1 | add table | SQL | | <span style="color: #339966">Pending</span> |
+-----------+---------+-------------+------+---------------------+---------+
The difference comes from the fact that PostgreSQL supports DDL transactions while others like H2 or MySQL don’t. As a result, PostgreSQL was able to rollback the transaction for the failed migration. Let’s see how this difference affects things when we try to repair the database.
3.2. Correct the Mistake and Re-Run the Migration
Let’s fix the migration file V1_1__add_table.sql by correcting the table name from table_one to table_two.
Now, let’s try and run the application again:
mvn spring-boot:run -Ph2
We now notice that the H2 migration fails with:
Validate failed:
Detected failed migration to version 1.1 (add table)
Flyway will not re-run the version 1.1 migration as long as an already failed migration exists for this version.
On the other hand, the postgre profile ran successfully. As mentioned earlier, due to the rollback, the state was clean and ready to apply the corrected migration.
Indeed, by running mvn flyway:info -Ppostgre we can see both migrations applied with Success. So, in conclusion, for PostgreSQL, all we had to do was correct our migration script and re-trigger the migration.
4. Manually Repair the Database State
The first approach to repair the database state is to manually remove the Flyway entry from flyway_schema_history table.
Let’s simply run this SQL statement against the database:
delete from flyway_schema_history where version = '1.1';
Now, when we run mvn spring-boot:run again, we see the migration successfully applied.
However, directly manipulating the database might not be ideal. So, let’s see what other options we have.
5. Flyway Repair
5.1. Repair a Failed Migration
Let’s move forward by adding another broken migration V1_2__add_table.sql file, running the application and getting back to a state where we have a failed migration.
Another way to repair the database state is by using the flyway:repair tool. After correcting the SQL file, instead of manually touching the flyway_schema_history table, we can instead run:
mvn flyway:repair
which will result in:
Successfully repaired schema history table "PUBLIC"."flyway_schema_history"
Behind the scenes, Flyway simply removes the failed migration entry from the flyway_schema_history table.
Now, we can run flyway:info again and see the state of the last migration changed from Failed to Pending.
Let’s run the application again. As we can see, the corrected migration is now successfully applied.
5.2. Realign Checksums
It’s generally recommended never to change successfully applied migrations. But there might be cases where there is no way around it.
So, in such a scenario, let’s alter migration V1_1__add_table.sql by adding a comment at the beginning of the file.
Running the application now, we see a “Migration checksum mismatch” error message like:
Migration checksum mismatch for migration version 1.1
-> Applied to database : 314944264
-> Resolved locally : 1304013179
This happens because we altered an already applied migration and Flyway detects an inconsistency.
In order to realign the checksums, we can use the same flyway:repair command. However, this time no migration will be executed. Only the checksum of the version 1.1 entry in the flyway_schema_history table will be updated to reflect the updated migration file.
By running the application again, after the repair, we notice the application now starts successfully.
Note that, in this case, we’ve used flyway:repair via Maven. Another way is to install the Flyway command-line tool and run flyway repair. The effect is the same: flyway repair will remove failed migrations from the flyway_schema_history table and realign checksums of already applied migrations.
6. Flyway Callbacks
If we don’t want to manually intervene, we could consider an approach to automatically clean the failed entries from the flyway_schema_history after a failed migration. For this purpose, we can use the afterMigrateError Flyway callback.
We first create the SQL callback file db/callback/afterMigrateError__repair.sql:
DELETE FROM flyway_schema_history WHERE success=false;
This will automatically remove any failed entry from the Flyway state history, whenever a migration error occurs.
Let’s create an application-callbacks.properties profile configuration that will include the db/callback folder in the Flyway locations list:
spring.flyway.locations=classpath:db/migration,classpath:db/callback
And now, after adding yet another broken migration V1_3__add_table.sql, we run the application including the callbacks profile:
mvn spring-boot:run -Dspring-boot.run.profiles=h2,callbacks
...
Migrating schema "PUBLIC" to version 1.3 - add table
Migration of schema "PUBLIC" to version 1.3 - add table failed!
...
Executing SQL callback: afterMigrateError - repair
As expected, the migration failed but the afterMigrateError callback ran and cleaned up the flyway_schema_history.
Simply correcting the V1_3__add_table.sql migration file and running the application again will be enough to apply to corrected migration.
7. Summary
In this article, we looked at different ways of recovering from a failed Flyway migration.
We saw how a database like PostgreSQL – that is, one that supports DDL transactions – requires no additional effort to repair the Flyway database state.
On the other hand, for databases like H2 without this support, we saw how Flyway repair can be used to clean the Flyway history and eventually apply a corrected migration.
As always the complete code is available over on GitHub.
