September 20, 2024

Policy Checks: Faster, safer, and easier database change management

See Liquibase in Action

Accelerate database changes, reduce failures, and enforce governance across your pipelines.

‍

Even though seatbelts were invented in the mid-19th century, they didn’t catch on with coaches and covered wagons – or even with much faster automobiles, until the middle of the next century.

What does this have to do with modern application development, data, and database management teams?

It illustrates the generally accepted concept that humans don’t tend to put safety first – we prefer speed. We dream about going faster and only consider safety once we’ve figured out how to go faster.

For technology to provide the most significant benefit to the broadest group of people possible, safety needs to become a primary concern. It also needs to be packaged conveniently and effectively to make it easy to integrate into our lives. In the case of seatbelts, that happened when vehicles finally traveled fast enough for humans to think, “that’s fast enough.” Then, they shifted to focus on how to stay safer while traveling at these high speeds.

Sound familiar?

For instance, application development pipelines can achieve peak-velocity release cadence of small, frequent deployments (aligned with DevOps philosophies) because of automatic safety features at critical checkpoints.

Now, think about database updates. In the change management workflow, is there anything akin to seatbelts that provide automatic, passive safety benefits? Chances are, database changes ride without even a lap belt – cruising to a database administrator’s (DBA’s) desk where they’re limited to a slow, tedious, and manual review process.

Just like a driver behind the wheel, the whole environment is put at risk when there’s so much potential and opportunity for human error. Database updates not only take too long to review and push through the pipeline, but often make it through DBA review only to cause unexpected problems in later stages – or critically, in production.

To bring the process closer to the speed and efficiency of the rest of the CI/CD pipeline and provide a customizable, trustworthy, and reliable safety check on every update, Liquibase offers Policy Checks as part of its complete database DevOps solution.

Find out how you can leverage Liquibase’s Policy Checks to:

Speed up database change management
Get immediate feedback on change requests
Reduce errors and deployment failures
Elevate database code quality
Support smaller, more frequent deployments

Policy Checks for database changes: a seatbelt for CI/CD

With all of the recent advancements in CI/CD and release automation, the focus is always, “Do more stuff faster with less hassle.”

This mantra works great for IT-driven innovation until your app won’t start, your performance tanks, or your database is MIA after an update. The long-term success of any investment in digital transformation depends on the ability to arrive at your destination safely.

The trick is ensuring safe arrival with something as easy as fastening a seat belt. That’s why Liquibase built the capability to automatically check database change requests for policy, as defined by best practices and custom organizational standards, and alignment with internal policies. Policy Checks for database changes are a powerful safety feature that is so simple to establish that it quickly becomes second nature — just like fastening a seatbelt.

Liquibase Pro’s Policy Checks analyze database change submissions – ChangeLogs, ChangeSets, and SQL – to identify any risky patterns, quality issues, access concerns, or other potential errors that can cause problems now or later in the pipeline. As part of a CI/CD pipeline, Policy Checks can be automatically executed as part of the automated change management workflow. They can also be executed on-demand, such as during development, to increase overall change submission quality.

Policy Checks empower database DevOps benefits and best practices including:

Risk mitigation: Policy Checks proactively identify and address potential errors or risky changes before they are implemented. This early detection helps avoid costly and time-consuming fixes in later stages of the development cycle.
Enhanced compliance: By ensuring that all changes comply with predefined standards and policies, Policy Checks uphold the integrity and security of your database environment. This is crucial for organizations that need to meet strict regulatory requirements.
Improved change quality: Policy Checks help maintain a high standard of change submissions by enforcing best practices and organizational policies throughout the change management process. This leads to improvements in the overall quality of the database changes.
Flexibility in execution: While Policy Checks can be automated within the CI/CD pipeline for continuous assessment, they can also be executed on-demand. This flexibility allows developers to perform checks during development phases, fostering a culture of quality and continuous improvement.
Streamlined development process: Integrating Policy Checks into the development process not only secures the database changes but also streamlines workflows by reducing the need for multiple revisions and backtracking. Developers can move forward with confidence, knowing their changes meet all necessary criteria.
Detailed feedback and reporting: When Policy Checks identify issues, they provide detailed feedback and actionable insights. This not only helps in quick resolution but also aids in educating developers, improving their understanding of best practices and compliance requirements.

Incorporating Policy Checks into your database change management strategy not only enhances security and compliance but also supports a more efficient and reliable development process.

With Liquibase’s Policy Checks, teams can achieve faster deployment cycles, minimize disruptions, and maintain a high standard of database integrity and performance.

Giving the database DevOps community what it wants

Policy Checks came into being as most things do at Liquibase — through partnerships with our Community and customers. According to a 2020 survey, the most valuable feature requested by Liquibase users was the ability to detect security, data integrity, and compliance issues in database changes as early in the pipeline as possible.

With this directive, Liquibase’s engineering team got to work, digging up an old enterprise rules engine capability for inspiration. To make the new feature as capable and user-friendly as possible, the team gathered direct feedback from more than 100 developers, DBAs, and DevOps professionals through interviews and surveys. This provided a solid understanding of their processes, the kinds of database changes that cause the most problems, and the best practices they have established to keep their databases available, stable, and performant.

The resulting Policy Checks are simple to use, inherently flexible, and reliably powerful. This database DevOps feature amplifies and accelerates the most important feedback loop in database development: the initial change approval process.

Policy Checks began as a tool for detecting changes that modify user and role privileges or introduce potentially destructive changes. It’s why, before teams configure their own Policy Checks, any initial checks will return alerts for any SQL scripts and changesets that alter privilege configuration, drop tables, drop columns, or modify data types. From this beginning use case, Policy Checks have grown to include capabilities that:

Enforce best practices, naming conventions, and data types
Enhance control and flexibility of changes
Improve tracking with granularity and context
Manage access with more granular privileges

How Policy Checks Work

For relational databases with advanced Liquibase support, Policy Checks are executable in two ways:

Automatically, as part of the change management pipeline
On-demand, during development or prior to submission

Either way, they are composed of three primary components:

Status and dynamic (customizable) checks – see the full List of Policy Checks – all of which can be enabled or disabled
The checks command and sub-commands
The checks settings file that defines your team/organization’s unique configurations and parameters

Policy Checks can analyze ChangeLogs and ChangeSets as well as the entire target database. The full list includes common checks like dropped column or table, missing labels, privileges, and more. Teams can configure multiple Policy checks to suit the needs of their pipelines, then combine them into Policy Check Packages for easy, comprehensive execution.

Then, those Policy Checks can be used on-demand during development and testing or set up as Automated Policy Checks integrated into the automated change management workflow and CI/CD pipeline.

Database and development teams love how easy it is to manage their Policy Checks. It’s simple to extend to the specific needs of the organization, team, or project. If any standard checks don’t appeal to your team, it only takes a single, simple command to disable them for all future uses. If you have more specific needs around the kinds of SQL statements allowed, a template check lets teams define patterns to look for in a flexible way.

Custom Policy Checks

While Liquibase Pro offers out-of-the-box Policy Checks that can provide immediate improvements, teams can also define their own Custom Policy Checks. They can also head to this GitHub repo to pick up customized Policy Checks developed by others in the Liquibase Community

You can create as many checks as needed. In fact, Liquibase worked closely with customers to test out real-world configuration scenarios during the development of Policy Checks. Their team provided us with a list of checks they wanted to run before committing a new change or during the integration build process for new changes. In about 30 minutes, Liquibase had completely configured all of their Custom Policy Checks.

Wit Custom Policy Checks, you can write Python-based checks for your guidelines, policies, and best practices. These fully customizable Policy Checks help ensure compliance for your organization’s specific tech stack and nuanced policies — if you can codify it in Python, it can be a Policy Check.

Integrating Policy Checks to enhance the whole CI/CD pipeline

Integrating Policy Checks into database change management significantly enhances the entire CI/CD pipeline by adding a layer of validation that ensures only safe, compliant, and optimized database changes are promoted through to production. Complex, dependent, and critical database elements and changes no longer need to slow down the pipeline or disrupt availability – they can be adequately tested and verified, then successfully deployed, in line with the application team’s strong, steady cadence.

Let’s break that down a bit – here’s how Policy Checks improve your pipelines.

Ensure Consistency and Reliability

Policy Checks standardize the validation of changes across all stages of the development lifecycle. By enforcing consistent standards from the initial stages of development to deployment, these checks ensure that all database changes are reliable and function as intended when they go live. This consistency helps prevent the common issue of "it works on my machine" by ensuring changes operate consistently across all environments, reducing the risk of failures after deployment.

Accelerates the Development Process

With automated Policy Checks, the process of reviewing and approving database changes becomes much faster. Developers no longer need to wait for manual reviews from DBAs, which can often be a bottleneck in the pipeline. By speeding up this process, the overall time from development to deployment is reduced, enabling faster release cycles and quicker delivery of features and fixes to users.

Reduces Downtime and Improves Availability

By catching potential issues early in the development process, Policy Checks minimize the risk of serious problems occurring in production, which can lead to costly downtime. Ensuring that changes are thoroughly vetted before they reach production helps maintain application availability, a critical factor for user satisfaction and business continuity.

Find out how to use Liquibase to correlate database changes to system availability with other capabilities: tracking and observability.

Enhances Security and Compliance

Integrating Policy Checks into the database change management process helps enforce security policies and compliance with regulatory standards. These checks can automatically scan for vulnerabilities or non-compliance with data handling standards, significantly reducing the risk of security breaches or legal issues associated with non-compliance. This is especially important for organizations in regulated industries such as finance, healthcare, and public services. By improving database compliance, Policy Checks safeguard the organization’s most critical asset – its data stores.

Facilitates Better Collaboration and Accountability

When Policy Checks are part of the CI/CD pipeline, every team member has visibility into the compliance and quality standards being enforced. This transparency fosters better collaboration between developers, DBAs, and operations teams, as everyone understands the criteria that changes must meet. Additionally, if issues do arise, the automated checks provide clear documentation and feedback, making it easier to pinpoint responsibility and resolve issues more efficiently.

This database observability capability also helps teams adopt a DevOps-minded continuous optimization approach for the change management workflow.

Provides Actionable Insights for Continuous Improvement

The data gathered from automated Policy Checks can be analyzed to identify trends, common errors, and areas for improvement. This ongoing analysis allows teams to continually refine their processes, improve the quality of database changes, and enhance the overall performance of the CI/CD pipeline. These insights are invaluable for long-term improvements and help organizations adapt their strategies to better meet their operational goals.

Liquibase Pro also provides Checks Run Reports that detail the activity surrounding a Policy Check or Policy Check Package. Useful for auditing and root cause analysis, these reports provide an in-depth forensic profile of the change and how it fared in the Policy Checks run against it.