One-Stop Guide to Database Migration with Liquibase and Spring Boot

Spring Boot provides integration with database migration tools Liquibase and Flyway. This guide provides an overview of Liquibase and how to use it in a Spring Boot application for managing and applying database schema changes.

Example Code

Why Do We Need Database Migration Tools?

Database migration tools help us to track, version control, and automate database schema changes. They help us to have a consistent schema across different environments.

Refer to our guides for more details on why we need database migration tools and for a quick comparison of Liquibase and Flyway.

Introduction to Liquibase

Liquibase facilitates database migrations with not only plain old SQL scripts, but also with different abstract, database-agnostic formats including XML, YAML, and JSON. When we use non-SQL formats for database migrations, Liquibase generates the database-specific SQL for us. It takes care of variations in data types and SQL syntax for different databases. It supports most of the popular relational databases.

Liquibase allows enhancements for databases it currently supports through Liquibase extensions. These extensions can be used to add support for additional databases as well.

Core Concepts of Liquibase

Let’s have a look at the vocabulary of Liquibase:

• ChangeSet: A changeSet is a set of changes that need to be applied to a database. Liquibase tracks the execution of changes at a ChangeSet level.

• Change: A change describes a single change that needs to be applied to the database. Liquibase provides several change types like “create table” or “drop column” out of the box which are each an abstraction over a piece of SQL.

• Changelog: The file which has the list of database changeSets that needs to be applied is called a changelog. These changelog files can be in either SQL, YAML, XML, or JSON format.

• Preconditions: Preconditions are used to control the execution of changelogs or changeSets. They are used to define the state of the database under which the changeSets or changes logs need to be executed.

• Context: A changeSet can be tagged with a context expression. Liquibase will evaluate this expression to determine if a changeSet should be executed at runtime, given a specific context. You could compare a context expression with environment variables.

• Labels: The purpose of Labels is similar to that of contexts. The difference is that changeSets are tagged with a list of labels (not expressions), and during runtime, we can pass a label expression to choose the changeSets which match the expression.

• Changelog Parameters: Liquibase allows us to have placeholders in changelogs, which it dynamically substitutes during runtime.

Liquibase creates the two tables databasechangelog and databasechangeloglock when it runs in a database for the first time. It uses the databasechangelog table to keep track of the status of the execution of changeSets and databasechangeloglock to prevent concurrent executions of Liquibase. Refer to the docs for more details.

Liquibase with Spring Boot

Now that we went through the basics of Liquibase let’s see how to get Liquibase running in a Spring Boot application.

Setting Up Liquibase in Spring Boot

By default Spring Boot auto-configures Liquibase when we add the Liquibase dependency to our build file.

Spring Boot uses the primary DataSource to run Liquibase (i.e. the one annotated with @Primary if there is more than one). In case we need to use a different DataSource we can mark that bean as @LiquibaseDataSource.

Alternatively, we can set the spring.liquibase.[url,user,password]properties, so that spring creates a Datasource on its own and uses it to auto-configure Liquibase.

By default, Spring Boot runs Liquibase database migrations automatically on application startup.

It looks for a master changelog file in the folder db/migration within the classpath with the name db.changelog-master.yaml. If we want to use other Liquibase changelog formats or use different file naming convention, we can configure the spring.liquibase.change-log application property to point to a different master changelog file.

For example, to use db/migration/my-master-change-log.json as the master changelog file, we set the following property in application.yml:

spring:
  liquibase:
    changeLog: "classpath:db/migration/my-master-change-log.json"

The master changelog can include other changelogs so that we can split our changes up in logical steps.

Running Our First Database Migration

After setting everything up, let’s create our first database migration. We’ll create the database table user_details in this example.

Let’s create a file with name db.changelog-master.yaml and place it in src/main/resources/db/changelog:

databaseChangeLog:
  - include:
      file: db/changelog/db.changelog-yaml-example.yaml

The master file is just a collection of includes that points to changelogs with the actual changes.

Next, we create the changelog with the first actual changeset and put it into the file src/main/resources/db/changelog-yaml-example.yaml:

databaseChangeLog:
  - changeSet:
      id: create-table-user
      authors: [liquibase-demo-service]
      preConditions:
        - onFail: MARK_RAN
          not:
            tableExists:
              tableName: user_details
      changes:
        - createTable:
            columns:
              - column:
                  autoIncrement: true
                  constraints:
                    nullable: false
                    primaryKey: true
                    primaryKeyName: user_pkey
                  name: id
                  type: BIGINT
              - column:
                  constraints:
                    nullable: false
                  name: username
                  type: VARCHAR(250)
              - column:
                  constraints:
                    nullable: false
                  name: first_name
                  type: VARCHAR(250)
              - column:
                  name: last_name
                  type: VARCHAR(250)
            tableName: user_details

We used the changeType createTable, which abstracts the creation of a table. Liquibase will convert the above changeSet to the appropriate SQL based on the database that our application uses.

The preCondition checks that the user_details table does not exist before executing this change. If the table already exists, Liquibase marks the changeSet as having run successfully without actually having run.

Now, when we run the Spring Boot application, Liquibase executes the changeSet which creates the user_details table with user_pkey as the primary key.

Using Changelog Parameters

Changelog parameters come in very handy when we want to replace placeholders with different values for different environments. We can set these parameters using the application property spring.liquibase.parameters, which takes a map of key/value pairs:

spring:
  profiles: docker
  liquibase:
    parameters:
      textColumnType: TEXT
    contexts: local
---
spring:
  profiles: h2
  liquibase:
    parameters:
      textColumnType: VARCHAR(250)
    contexts: local    

We set the Liquibase parameter textColumnType to VARCHAR(250) when Spring Boot starts in the h2 profile and to TEXT when it starts in the docker profile (assuming that the docker profile starts up a “real” database).

We can now use this parameter in a changelog:

databaseChangeLog:
  - changeSet:
     ...
      changes:
        - createTable:
            columns:
             ...
              - column:
                  constraints:
                    nullable: false
                  name: username
                  type: ${textColumnType}

Now, when the Spring Boot application runs in the docker profile, it uses TEXT as column type and in the h2 profile it uses VARCHAR(250).

Using Liquibase Context

As described earlier, context can be used to control which changeSets should run. Let’s use this to add test data in the test and local environments:

<databaseChangeLog>
 <changeSet 
   author="liquibase-docs" 
   id="loadUpdateData-example" 
   context="test or local">
   <loadUpdateData
     encoding="UTF-8"
     file="db/data/users.csv"
     onlyUpdate="false"
     primaryKey="id"
     quotchar="'"
     separator=","
     tableName="user_details">
   </loadUpdateData>
 </changeSet>
</databaseChangeLog>

We’re using the expression test or local so it runs for these contexts, but not in production.

We now need to pass the context to Liquibase using the property spring.liquibase.contexts:

---
spring:
  profiles: docker
  liquibase:
    parameters:
      textColumnType: TEXT
    contexts: test

Configuring Liquibase in Spring Boot

As a reference, here’s a list of all properties that Spring Boot provides to configure the behavior of Liquibase.

Enabling Logging for Liquibase in Spring Boot

Enabling INFO level logging for Liquibase will help to see the changeSets that Liquibase executes during the start of the application. It also helps to identify that the application has not started yet because it is waiting to acquire changeloglock during the startup.

Add the following application property in application.yml to enable INFO logs:

logging:
  level:
    "liquibase" : info

Best Practices Using Liquibase

• Organizing Changelogs: Create a master changelog file that does not have actual changeSets but includes other changelogs (only YAML, JSON, and XML support using include, SQL does not). Doing so allows us to organize our changeSets in different changelog files. Every time we add a new feature to the application that requires a database change, we can create a new changelog file, add it to version control, and include it in the master changelog.

• One Change per ChangeSet: Have only one change per changeSet, as this allows easier rollback in case of a failure in applying the changeSet.

• Don’t Modify a ChangeSet: Never modify a changeSet once it has been executed. Instead, add a new changeSet if modifications are needed for the change that has been applied by an existing changeSet. Liquibase keeps track of the checksums of the changeSets that it already executed. If an already run changeSet is modified, Liquibase by default will fail to run that changeSet again, and it will not proceed with the execution of other changeSets.

• ChangeSet Id: Liquibase allows us to have a descriptive name for changeSets. Prefer using a unique descriptive name as the changeSetId instead of using a sequence number. They enable multiple developers to add different changeSets without worrying about the next sequence number they need to select for the changeSetId.

• Reference data management: Use Liquibase to populate reference data and code tables that the application needs. Doing so allows deploying application and configuration data it needs together. Liquibase provides changeType loadUpdateData to support this.

• Use Preconditions: Have preconditions for changeSets. They ensure that Liquibase checks the database state before applying the changes.

• Test Migrations: Make sure you always test the migrations that you have written locally before applying them in real nonproduction or production environment. Always use Liquibase to run database migrations in nonproduction or production environment instead of manually performing database changes.

Running Liquibase automatically during the Spring Boot application startup makes it easy to ship application code changes and database changes together. But in instances like adding indexes to existing database tables with lots of data, the application might take a longer time to start. One option is to pre-release the database migrations (releasing database changes ahead of code that needs it) and run them asynchronously.

Other Ways of Running Liquibase

Liquibase supports a range of other options to run database migrations apart from Spring Boot integration:

• via Maven plugin
• via Gradle plugin
• via Command line
• via JEE CDI Integration
• via Servlet Listener

Liquibase has a Java API that we can use in any Java-based application to perform database migrations.

Conclusion

Liquibase helps to automate database migrations, and Spring Boot makes it easier to use Liquibase. This guide provided details on how to use Liquibase in Spring Boot application and some best practices.

You can find the example code on GitHub.

We also have a guide on using Flyway, another popular alternative for database migrations.