5 Challenges for DBAs Making Database Changes

There is no doubt that CICD and DevOps are here to stay. Companies know that they need to automate the software release process. Liquibase co-founder and CTO, Robert Reeves, even talks about automation as a survival strategy. Liquibase’s Solutions Architect, Dan Zentgraf, speaks extensively about databases in CI/CD in his multi-part blog series. In this blog, I want to address the specifics behind why it’s important to automate database changes and help you understand more about the challenges that DBAs face where automation can help.

DBAs are critical

DBAs are a critical part of any enterprise. They ensure that the databases underlying applications are optimized for data and performance.

If you ask people at your company to describe what your DBAs do throughout their day, you’ll likely receive a variety of answers including:

• database design
• installing databases
• query support
• review scripts
• database optimization
• backing up data
• system monitoring
• database deployments

They are responsible for a whole host of tasks. One of these tasks is to review database scripts and deploy them. Without the right tools and database release automation, script reviews and database deployments are very manual DBA operations. As a result, DBAs are inundated with requests to make database changes and deploy them to environments.

How difficult is it to deploy a database script?

Let’s start with the basics: How difficult is it to deploy a database script anyway? It’s actually not difficult at all. A DBA can deploy a database change to the database in less than a minute. So what really makes database changes so cumbersome? There are five reasons that this task gets difficult in a hurry.

5 key reasons database changes are so difficult

1. Several roles are involved

Let’s take a look at which roles are involved with authoring database changes. First, the software architects or product owners plan the new features. They collaborate with several roles to make database changes. They work with data architects to come up with how to structure database changes. This creates DDL (Data Definition Language) scripts. Developers then write code — PL/SQL scripts — based on those database structures. Business analysts then insert data into these structures. So now there are three types of changes authored by three different personas:

• DDL scripts authored by data architects
• PL/SQL scripts authored by developers
• Data scripts authored by business analysts

2. Change request overload

A recent study shows that 57% of all application changes require a corresponding database change. We learned the first reason why database changes are difficult is that there are three types of changes that need to be deployed to a database. Since over half of application updates require a database change and there are three types of changes, we’ve already multiplied the number of change requests DBAs receive. However, there is also not just one database. There are a series of databases (e.g., DEV, TEST, STAGE and PROD).

Changes are first introduced into the DEV database and then gradually moved into production by series of deployments into intermediate environments. Script authors generally don’t have access to all of these environments (DEV environments could be made accessible to script authors). For deploying to non-DEV environments, the scripts are sent to DBAs via change requests. Each change request contains information on what to deploy (the script) and where to deploy it (target environment).

And then there’s the rework. In a recent study, 91% of DBA respondents said that they have to rework database changes multiple times before they are ready for production deployment. Reworking changes may add 5+ times as many deployments before settling on final changes. You can see how quickly a DBA’s workload can multiply.

3. SLAs and the developer wait state

Since DBAs are the ones with access to non-DEV environments, they are responsible for pushing all changes to these environments. When a change request arrives at a DBA’s desk, it may take up to 24 hours for the changes to be deployed. Some organizations have their DBAs offshore where the turnaround time could be higher due to time zone differences. Developers don’t like to wait this long. They follow up with DBAs on Slack channels to prioritize their change requests. DBAs are under pressure to deploy change requests as quickly as possible. When rework is needed, then those change requests also arrive fast and furious.

Since the process is manual, the sheer number of requests with the added pressure to get the database updates turned around quickly often leads to mistakes. These mistakes can lead to downtime and more rework.

4. Late nights or weekend deployments

Deployments to PROD typically require further coordination. Many companies require all script authors, as well as their managers, to be on the conference call as they perform PROD deployments in case something goes wrong so that all parties are available to troubleshoot and hotfix. These deployments are generally scheduled at off-peak hours which means late at night or weekends for everyone involved.

5. DB to DBA ratio

Industry ratio, as well as personal conversations, have revealed that the average database-to-DBA ratio varies wildly from organization to organization. It can range from 40:1 to 2,000:1. Yes, that means as many as two thousand databases (prod and non-prod) per DBA!

Automation can help DBAs

When database changes and releases are automated, a DBA doesn’t have to spend most of their time reacting to change requests and coordinating PROD release activities. Some may say that these are typical DBA responsibilities. At Liquibase, we say get away from all of the manual work and let’s elevate DBA responsibilities. Instead of focusing on deployments, let’s enable them to do script reviews just as developers perform code reviews prior to merging their code. DBAs will learn source code to perform code review, mostly to catch bad database practices, prior to handing the automation tool to deploy the script. But most of all, let’s keep DBAs focused on keeping databases running efficiently.

Interested in making sure your DBA team can focus on the most important tasks? Contact us and we’ll find the best database release automation workflow that will fit your needs.