CI/CD tools have become an integral part of any team’s software development cycle. According to senior analyst Chris Condo from Forrester, “Continuous integration tools have evolved from simply automating source code merges to become developer workflow engines.” Chris also notes that CI/CD tools “keep the dev team running at peak performance by orchestrating all of the automation that conveying code from check-in to test to deploy requires.”1
But why are cloud-based CI/CD tools growing in popularity in particular? One reason is they remove the need to dedicate people and resources to install, operate, and maintain CI/CD infrastructure. This in conjunction with the ability to configure a CI/CD tool (on-premise or in the cloud) to enforce workflow lets teams focus on developing and deploying fast while still adhering to security and compliance requirements.
We’ve seen this popularity first hand as companies like Hitachi Vantara, iRobot, Lufthansa Industry Solutions, Rappi, and Schneider Electric have adopted our cloud-based CI/CD tool, Bitbucket Pipelines. Adoption of Pipelines has accelerated since its launch 3 years ago, and customers now test their code almost 750,000 times a week, using over 4.3 million minutes in build time per week.
So we know there is appetite in the market for teams to adopt or migrate to a cloud-based CI/CD tool, but how do you know what to look for? Here are 10 things to consider.
Out-of-the-box integrations with your code
There are two flavors of cloud-based CI/CD tools you can choose from – a standalone variety or one integrated with your source code management tool. Both remove the overhead of having to provision and manage physical infrastructure, but CI/CD tools integrated out of the box with your code means there’s no finicky integration to set up, repositories to synchronize and, best of all, user management to configure and manage twice.
We built Pipelines inside Bitbucket Cloud specifically to take advantage of this and more (and you only get billed once for both your SCM and CI/CD tool).
Easy setup and configuration
On-premise CI/CD tools can be difficult to set up and configure, and a nightmare to maintain and update. As developers take on more responsibility in the setting up and configuring of their build environments, it’s important that teams consider a tool that makes it easy to set up and get started.
With Pipelines we rely on Docker to provide isolated and reproducible builds. It’s simple to get started or to replicate your test, staging, or production environments. It’s a 2-step process to configure a pipeline, and a number of templates for languages (e.g. Javascript, Python, Go etc.) are available to get started. Configuration as code means your build configuration is versioned, just like any other file in your repository, and lets you reproduce your build from any point in time.
Increased visibility and collaboration
As more organizations adopt DevOps approaches to development it’s vital that all teams have visibility into what’s being deployed to customers. Not only that, the information should be available in a context that makes sense for what you’re doing at the time – too often teams need to context switch and dig around to find the information they need in a separate tool.
Pipelines for example has integrations with tools like Jira, Slack, and Microsoft Teams that provide context on your builds and deployments where your team plans and collaborates. For collaboration tools like Slack in particular, it’s easy to see what’s happening with your CI/CD tool and act on it too, giving your team the power to perform key review and deployment actions right where they communicate on a regular basis.
Jira Software integration
Integration with a tool that provides a way to track and coordinate all the work activities that go into modern software development is important, especially in a DevOps world where all teams need visibility into work ongoing and completed. Jira Software is the number one software development tool for agile teams, who use it to keep their business running, alert internal stakeholders of progress, and rely on automation to keep Jira up to date.
We’re proud that Pipelines offers the most feature-rich and robust Jira Software integration of any CI/CD tool, providing visibility for everyone who works inside Jira from backlog all the way through to deployment. With the integration you can:
- See which Jira issues are included in a deployment in Pipelines
- See the status of builds associated with a particular Jira issue
- View where a Jira issue has been deployed to inside Jira Software
Integrations and extensibility
Whether specific requirements need to be met or not, every team has their own way of working and need the freedom to choose the tools they want to get the job done. It’s important then to consider a CI/CD tool that not only does what you need it to do but integrates easily with the other tools you use.
We recently announced Pipes, a way to connect Pipelines to other tools to build powerful CI/CD workflows. Pipes are supported by the vendor so you never have to worry about maintaining them and it’s easy to use in a plug and play fashion. Best of all, it’s easy to write your own pipes to build and connect the tools you use just the way you want.
Build and deployment environment support
Similar to there being no one-size-fits-all solution for the tools teams use, teams have a variety of requirements and preferences when it comes to build and deployment environments. Consider a CI/CD tool that supports the operating systems you need (e.g. Linux, Mac, Windows, Android etc.),and offers the flexibility to deploy to different cloud vendors like AWS or Azure.
For example, Pipelines supports Linux and .NET Core build environments but not .NET Framework or macOS yet, and similarly for mobile developers Android is supported but not iOS yet. Pipelines lets you deploy to whichever cloud provider you use. For the more common providers like AWS or Azure, there are supported pipes to make this process easier, but it’s also easy to build your own pipe and integration with other providers.
Security and compliance
Some organizations are hesitant to adopt cloud-based tools citing lack of security and protection of their data, but these days many cloud CI/CD vendors adhere to widely accepted standards and regulations. For example Pipelines is built on top of Bitbucket Cloud whose data is stored in our secure cloud that is compliant with SOC2 type 2, SOC3, ISO27001 and ISO27018. And Pipelines data specifically is stored encrypted at rest on the storage medium.
Cloud CI/CD vendors also offer features that enable teams to adhere to compliance regulations too, such as required 2-factor authentication and IP whitelisting for access that Bitbucket provides.
Workflow flexibility
It's important to find a CI/CD tool that is flexible enough to suit your team's workflow and enable you to adhere to your specific security and compliance requirements. In an ideal world the tool automatically enforces these requirements, allowing your team the freedom to move as fast as possible to deliver value to your customers.
Workflow flexibility is core to Pipelines. An advantage of being fully integrated inside Bitbucket is leveraging its merge checks to ensure code is reviewed and builds pass before being merged to master, protecting the quality of code that eventually gets deployed to production. Specifically for deployments, features like manual deployments and deployment permissions give teams granular control over who can deploy code to production and when.
Performance, uptime, and scalability
Performance, uptime, and scale are important when evaluating any product either on-premise or in the cloud, and your CI/CD tool is no exception – it doesn’t matter how many features it has if you can’t access it half the time! Understand how these cloud-based CI/CD tools scale in response to load and identify ways the tool helps in accelerating builds.
For comparison’s sake, Pipelines has an average uptime of ~99.97% over the last 12 months, is designed to auto-scale build agents based on load, and methods such as caching and parallel steps can be used to help speed up build times and give teams feedback faster.
Price
Compared to on-premise alternatives, cloud-based CI/CD tools eliminate the cost of acquiring, maintaining, and updating infrastructure and the dedicated teams needed to look after them. There are different ways cloud vendors charge for usage of their CI/CD tool, and it’s wise to understand what costs might look like for your team.
Pipelines pricing is based off a simple, consumption-based model of build minutes used, and every Bitbucket plan includes build minutes. Unlike other cloud vendors we don’t charge for concurrency, meaning you don’t pay extra to follow CI/CD best practice and run your pipelines steps as fast as you can.
Choosing the right cloud CI/CD tool for you
Every team should have a CI/CD tool as part of their development toolchain, and now's as good a time as any to evaluate a cloud-based one. While we’ve listed 10 things to consider when looking for your ideal tool there are likely others more specific to your situation to weigh up and prioritize too.
Whatever your requirements may be, a tool like Pipelines might be perfect for your needs, so try it free and see it in action yourself.
Best of luck!
Want to ensure only the right members of the team can merge code and deploy to production? Set up deployment permissions and required merge checks with our Premium plan.
1 Cloud CI Tools Mature To Meet Enterprise Challenges, Christopher Condo, with Christopher Mines, Andrew Dobak, Andrew Reese, Forrester, June 26, 2019
