Automate Coverage Reports in Pull Requests with Bitbucket, Jenkins and SonarCloud

This is a guest post by Alwyn Davis, Senior Software Developer at Instaclustr

At Instaclustr, we've experienced significant growth in our team sizes that has been great for increasing the scope and speed of our development.  The flip-side to this benefit is that with the increased velocity of projects came increased pressure on approvers to take time from their own tasks, and provide quality feedback at a faster pace.

To address this, we overhauled our existing build systems to:

• Automate static code analysis
• Expose important metrics (such as test coverage, whether tests have passed); and
• Expose it to reviewers within pull requests

Now, our review workflow is:

1. Developer creates a PR in Bitbucket, targeting the release branch
2. Jenkins sees the creation of the PR and starts our build-and-test pipeline beginning with unit and system tests.  If successful, the pipeline progresses through to our end-to-end tests. At each stage, coverage results are forwarded to SonarCloud for analysis
3. When an approver views the PR, Bitbucket (via the SonarCloud widget) pulls in the code analysis results and provides relevant context

Approvers can immediately know:

• Coverage of new code code in the PR
• If there are any common code errors (e.g. not closing resources)
• That style guidelines have been followed (e.g. how deep is the inheritance tree)

Best of all, relatively few changes were required to implement this!

1. Jenkins

We use Jenkins as our build system, so we created a multibranch pipeline job that uses the Bitbucket Branch Source Plugin to poll for any new or updated PRs targeting our release branch.  The pipeline trigger can then be configured to scan every minute.

Once triggered, the job will run our test pipeline Jenkinsfile.

The relevant parts of our Jenkinsfile are:

• Generating coverage reports using the Jacoco plugin
  

mvn clean org.jacoco:jacoco-maven-plugin:prepare-agent test --fail-at-end -DskipTests=false -am

• Uploading the generated reports to SonarCloud

mvn sonar:sonar --batch-mode --errors " +
"-pl ${context.env.TEST_MODULES} -am " +
"-Dsonar.projectKey=${Constants.SONARCLOUD_PROJECT_KEY} " +
"-Dsonar.organization=${Constants.SONARCLOUD_ORGANISATION} " +
"-Dsonar.verbose=true " +
"-Dsonar.host.url=${Constants.SONARCLOUD_URL} " +
"-Dsonar.login=${context.env.SONARCLOUD_TOKEN} " +
"-Dsonar.pullrequest.branch=${context.env.BRANCH_NAME} " +
"-Dsonar.pullrequest.base=${Constants.RELEASES_BRANCH} " +
"-Dsonar.pullrequest.key=${context.env.CHANGE_ID}

2. SonarCloud

Uploaded reports will automatically register a new PR in SonarCloud and can be explored through the SonarCloud Console to show inline views of code issues, test coverage trends and whether the PR meets customisable Quality Gates.  It's important to note that these metrics are calculated against new code introduced by the PR, so developers don't have to sort through an analysis of the entire codebase.

3. BitBucket

At this point, we have Jenkins automatically testing PRs and SonarCloud providing analysis.  To make it as easy as possible for approvers to see that information, we just need to enable the SonarCloud widget in our Bitbucket repository.

Now, whenever a PR is created, the widget will pull in test coverage, bugs and code smell metrics.  Using the Bitbucket Jenkins plugin also means that our PRs will show a handy "build passed" status to let us know when a branch has is successfully passing all test cases.

Summary

For Instaclustr, this setup has:

• Simplified the basic checks a reviewer would undertake by providing summary metrics for assessing the quality of a PR
• Highlighted test cases that don’t work in a parallel setting (e.g. incorrect usage of singleton objects)
• Highlighted poor test verification patterns (e.g. count pre- and post-test comparison rather than checking for presence of record)
• Code coverage highlights when tests aren’t being run (e.g. names not matching Surefire inclusion patterns)
• Ensures highlighting of simple code errors that can have significant impacts, such as not closing resources

Hopefully this has provided a quick guide to getting up and running with static code analysis; if you have any feedback, ideas or questions about this article, I would love to hear it!  

Author bio: Alwyn Davis is a Senior Software Developer at Instaclustr where he has worked on multiple infrastructure and development projects, in addition to providing client-facing support and delivering consulting projects. He has focused on the development of Instaclustr’s client-facing management systems implementation of Cassanda, Spark, and Kafka deployment processes.  He also has a background in technical consulting experience in search engine, database and CRM implementation, management and application development.

Love sharing your technical expertise? Learn more about the Bitbucket writing program.

Scaling your Bitbucket team? Upgrade your plan here