Database as Code: The foundation of database DevOps

If an organization’s data, and therefore its databases, are so critical to the business, why aren’t databases treated as first-class software citizens? Modern CI/CD pipelines streamline everything – until it’s time for database updates. 

Stuck in legacy configurations and cumbersome workflows, databases might be one of the most valuable assets, but they tend to get picked last when it comes to innovations and integrations that streamline and modernize these pipelines. 

This final, critical stage of the release process is still handled manually by database administrators (DBAs). It’s a workflow that’s slow at best and riddled with risks, vulnerabilities, and opportunities for error. 

But what if it didn’t have to be that way?

Treating the database as code aligns it with the rest of the fast, safe, collaborative, and automated application pipeline. This simple yet foundational change finally extends DevOps to the database, along with all its benefits. 

What is database as code? 

Database as code is the concept of managing data store structure, or schema, with automated scripts instead of manual configuration. Similarly, database change as code takes the same approach to orchestrating schema and data updates through automated pipelines, including review, testing, and deployment. 

A foundational element of database DevOps and extending CI/CD to the database, database as code aims to script, automate, and manage database updates with the precision, agility, and speed of application code updates. It allows the database to be treated with the same level of control, ensuring that database changes are as dynamic, flexible, and streamlined as application changes. 

Database as code primarily concerns the management of database schemas, configurations, and changes using version control systems. This practice treats database schemas and transformations as if they were application code. Changes to the database — such as schema migrations, updates, and rollbacks — are scripted and applied through automated processes, enabling consistency, version tracking, and the application of DevOps best practices and CI/CD integration.

This approach allows teams to apply systematic, repeatable, and reliable processes for database changes, reducing manual errors and increasing deployment speed and frequency.

That’s not to say shifting to a database-as-code approach is without challenges. Due to their foundational, stateful nature, and their vast interconnections, databases are rife with complexity not present on the application side. These must be reckoned with. 

Database changes present unique challenges

Databases are complex entities that store a wide array of data types, including structural, reference, and application data. Given this complexity, it's not surprising that changes to databases often come from multiple directions and for various reasons, which can make the concept of managing databases as code seem daunting. 

Changes might be prompted by application updates, increases in data volume, policy adjustments, performance enhancements, or even routine upgrades and patches. Each might originate from different parts of the organization — whether development, business units, or operations — placing DBAs in a precarious spot managing change review, team collaboration, and database integrity. Scalability, inherent to CI/CD pipelines, isn’t easily handled at the database layer, either. 

Database changes can have massive, far-reaching impacts, too, well beyond a buggy application experience. Since databases serve as the backbone of IT infrastructure, any errors or issues can lead to severe consequences, including degraded performance, data loss, or corruption. The repercussions of these problems are not just technical but also resource-intensive, taking up considerable time and cost to detect and rectify. 

A partial reaction to this conundrum is the industry’s shift from merely tracking the Mean Time Between Failures (MTBF) to emphasizing the Mean Time To Repair (MTTR). This shift acknowledges that while failures are inevitable, the critical measure of effectiveness is how swiftly and effectively an organization can respond and rectify these issues. This thinking gets closer and closer to database as code and the embrace of a pipeline that includes automation and immediate feedback. 

Avoid confusion with Infrastructure as Code (IaC)

The concepts of "Database as Code" and "Infrastructure as Code" (IaC) both revolve around managing and provisioning IT resources through code, but they focus on different aspects of the technology stack. Infrastructure as Code has come to be a common concept in DevOps – it dates back to the ‘70s – while database as code is a more recent development (2016) alongside the rise of database DevOps. 

Database as code, as the name implies, focuses on data stores. It manages database structure (schema), configurations, and changes to schema or data. But Infrastructure as Code focuses on computing technology like physical servers, virtual machines, network devices, and software-defined networks. Instead of physically configuring hardware or using tools to do it, machine-readable definition files handle the setup. 

While both practices leverage version control and automation, IaC deals with the underlying platforms and systems that host applications and data stores. Both are crucial for modern cloud environments and help in achieving high levels of operational efficiency and reliability, but are used by different teams. 

IaC automates infrastructure setup by scripting, or imaging, environments from the networking layers all the way up to the application layers. This leads to faster deployment, consistency across environments, scalable systems, and reduced management overhead. As you might expect, similar benefits come to the database when teams adopt a database as code approach, too. 

Core principles of database as code

Introduced by Dan North on the third slide of this presentation (pictured), the concept of treating database as code elevates data stores to their rightful place as first-class software citizens. 

As Liquibase Co-Founder Robert Reeves puts it in this article, now nearly a decade old: 

“Yes, I’m sold on DevOps; the entire database team is as well. Frankly, we’ve been waiting for the time when the tooling would allow us the same flexibility we see our system administrators and release managers enjoying.”

So how come, since 2016, when Reeves quipped that, “These days, DevOps as a concept is almost a foregone conclusion for IT organizations,” it has still not permeated the database change management pipeline?

DBAs aren’t going to hand over the keys to database deployments just because developers want to be able to push their changes right to production. There’s way too much at stake, including the digital foundations of the business. 

Addressing this problem means aligning development and databases at a deeper level. Perhaps, if they share a core functional commonality – built and driven through code – that allows the same kind of rigorous testing and safeguards application teams are used to, databases can join the DevOps fold. 

In a webinar hosted by Liquibase and DevOps.com, then-data architect for GE Transportation Chris Lawther stated:

“What we needed to do was to change our mindset of how we treated our database. We had to stop treating it like some special artifact or some unique scenario, and we started looking at it through the same perspective that we were treating our web code.”

And in a much more concise manner, Liquibase Co-Founder Pete Pickerill thinks of database code in terms of “Commit it and forget it.”

Why shouldn’t teams be able to check in SQL scripts, just like code, and ensure that they will automatically get tested and deployed? That’s the purest form of automation that North posits is necessary for treating the database as code. 

The portal and process for checking scripts and database changes into source code control should be easy enough for a developer to handle, not just a DBA. This gives DBAs some serious relief, as well as the ability to actually share and delegate responsibility for database deployments.

Without this kind of database management evolution, DBAs are stuck scrambling at the end of the automated application development pipeline. Of all the advancements in DevOps and CI/CD tools and methodologies, the last mile of the pipeline – managing database changes – still falls on the DBA to manually review, test, and implement. 

DevOps needs to embrace the database and embracing database as code is the first step. Otherwise, “the last 10 percent of the development-to-delivery process takes 90 percent of the time and resources.” And any effort to accelerate would mean neglecting the DBA’s arduous yet critical review process, which risks breaking the application – obviously, a non-starter. 

Without database as code and a rigorous DevOps process to manage it, any error that does make it into production is difficult to track down and understand. That’s especially problematic when the problem causes the business to lose money or frustrate its users. Even once its found, the error can be buried in a deployment and take weeks to sort out, roll back, redeploy, or fix forward. 

Database as code accelerates release cycles by making database deployments as agile, flexible, collaborative, safeguarded, and observable as the application. It equips the database change management process with:

• Version control and automation
• DevOps best practices
• CI/CD tech stack integration
• Easy rollback support
• Change monitoring and pipeline analytics (observability)

Version control for database code

Knowing its critical role for application development, it’s clear how database code version control is a core tenet of database DevOps. Version control for database code involves systematically managing changes to the database's schema and data scripts to maintain consistency, track changes, and facilitate collaboration within teams.

Database version control works by applying version numbers to different states of a database schema, allowing teams to track each change precisely and revert to previous versions if needed. This system is akin to using version control systems like Git for application code but is adapted to handle the unique requirements of databases, which are stateful and require careful management of data and schema changes. Think of it as Git for databases. 

Version control ensures that database changes are as agile and reliable as application code changes. With version control, database deployments can be aligned more closely with continuous integration and delivery (CI/CD) processes. This integration helps to speed up release cycles, reduces the risks associated with database changes, and allows databases to be treated with the same level of control and automation as the rest of the software development lifecycle.

The team can also look to database versioning for a trail of breadcrumbs leading from a production error back to a root cause. Traceability and observability improve and simplify the team’s ability to audit, analyze, and troubleshoot.

Integrating database code into the DevOps lifecycle

Database version control is foundational, yet it represents only part of the integration story within the DevOps lifecycle. By treating database changes as code, teams automate testing, validation, and deployment, leading to faster, more secure, and more reliable releases.

This integration is facilitated by version-controlled database change artifacts that are logically packaged and ready to be pushed through the pipeline. It begins with the automation of testing and validation for database changes. Using continuous integration tools, database code undergoes rigorous automated tests against various conditions and configurations. This proactive approach catches issues early, significantly reducing the risk of production problems and ensuring a high standard of quality and stability. Such automation not only streamlines processes but also frees up valuable human resources from repetitive and error-prone tasks.

Once a database change passes review, it smoothly transitions through the deployment stages — from QA to production. Automated systems update each environment with the latest version of the database code, maintaining consistency and minimizing manual efforts needed to synchronize environments. If an issue arises, the system triggers an alert early in the pipeline, allowing developers or DBAs to address and redeploy corrections before they escalate.

By incorporating these automated and version-controlled processes into the DevOps lifecycle, organizations achieve a more resilient and adaptive database management strategy. This strategy not only accelerates deployment cycles but also enhances overall operational efficiency, paving the way for a robust, always-on database infrastructure that supports continuous delivery and high performance.

Making the shift to database as code

Planning for full integration between database as code and IaC is great, but for those still fresh to database DevOps, this is too much of a leap. First, teams need to understand how to adopt database as code, including solving whatever challenges block the path. 

Coming up with a database as code framework and process often leaves teams floundering, though. To implement this program from scratch would be a massive lift – and that could also require massive investment. That’s why plenty of teams end up with a bespoke solution that’s workable but not optimal. Or they abandon the initiative, reverting back to the old, manual database change process. 

And what if management doesn’t get it? How can teams secure the resources they need to pull off this shift in database management? They might need to build a business case for database DevOps to convince leadership of its value. 

Database teams typically face two common scenarios. First, there may be a sense that the change management process isn't necessarily broken – so why invest in a fix? Second, it might seem more logical to throw more humans at the problem – but that’s only adding more complexity to a process that already struggles with clarity, collaboration, and efficiency. 

‍

What’s broken is database deployment velocity. It lags so far behind application releases that it creates a problematic gap delaying application deployments as database changes trudge through the antiquated workflow. 

That’s where a database DevOps platform like Liquibase comes in. It provides a framework for database as code, plus the automation, governance, and observability capabilities teams need to collaborate, test, and deploy at full speed and with complete confidence. 

With Liquibase, teams can fully embrace database as code, enable self-service database deployments, and get immediate feedback on any problematic changes. Given Liquibase’s unique Structured Logging that provides granular tracking for every operation, plus in-depth change monitoring and pipeline analytics (database observability), teams can seamlessly submit, track, revise, and deploy changes with the same fast pace as the rest of the DevOps pipeline. 

Teams using Liquibase can finally release database updates in small, frequent iterations (in line with DevOps best practices) with the added confidence of easy Targeted Rollbacks, customized Quality Checks, database Drift Detection, and other advanced DevOps features. For more detail, learn How Liquibase Works.  

Are you ready to start treating database code more like app code so you can “commit it and forget it”? Get started with a free trial or interactive demo of Liquibase Pro. Discover the power of database as code through the leading database DevOps solution with a comprehensive set of advanced automation, governance, and observability features. 

‍

Updated May 2024