Bitbucket helps teams build better software through collaboration. Our customers want tools they can trust and software that won't break under stress or load. Robustness, reliability, and security are very important attributes of our software. But, like any other software, sometimes things may go wrong. Anticipating those situations and building good diagnostics and recovery features in the product is crucial.
Quality at Bitbucket
Our Bitbucket team doesn’t include an army of dedicated testers who would spend weeks before the release to find bugs. That's inefficient and creates a sub-optimal split of responsibilities that leads to poor quality. Instead, we take a very different approach. Every developer on the team is responsible for the quality of the software. But not every developer is good at testing. That’s why we have a very small team of QA engineers whose primary goal is to enable every developer to do a better job with quality. It is usually achieved through the following tasks:
- Developer education: Help the development team learn best practices for writing high-quality code.
- Risk assessment: Think through all the possible scenarios that can cause the software to break and ensure that developers are aware of the highest-risk areas.
- Trend analysis: Analyze bugs and support requests to spot scenarios that have caused problems, and find ways to prevent similar problems in the future.
- Tool development: Provide test environments and new tools that help the development team ship high-quality software.
That's why QA in our team stands for "quality assistance" and not "quality assurance".
The challenges of scale
But it's not that simple, especially if you're building high-quality products at ever-increasing speed and scale with a small QA team. As Adrian Cockroft once said "scale breaks hardware, speed breaks software and speed at scale breaks everything!". There are tons of problems to solve. Most importantly, there is a dire need for innovative approaches and tools.
Good engineering practices
In order to achieve our goals of building quality at scale we use some of the following tactics, which we are happy to share with the Bitbucket community:
- Branching model – Follow a branching model using git that enables the use of pull requests for code reviews. Having a second pair of eyes looking at every code change helps spread the knowledge within the team and spot potential problems early.
- Code documentation – Write code that is self-documenting and as readable as possible. This is crucial for effective collaborative work on the same codebase and helps new starters get up to speed with the code base quickly.
- Comprehensive tests – Create unit tests, system-level integration tests and performance tests to verify the functionality under load and varying environments and scenarios.
- Internal Dogfooding – Demo new features to a large audience of internal users to solicit feedback from different software teams.We sometimes use new features and components for months internally before we release them to our customers.
- Feature flags – Implement new features behind flags that allow us to roll them out slowly, starting with a small group of internal users at Atlassian, and then widening the audience to a segment of Bitbucket users.
- "Blitz" testing – Use a coordinated, short (~1-2 hour) round of exploratory testing of a new product or feature(s) by a group of people from across the company. This gives us extra confidence that we definitely handled all those different use cases.
- Right tools – Use static analysis tools to detect common coding problems. Track exceptions in real time in all of our environments and measure everything to ensure the entire system is working as expected.
This is just the tip of the iceberg. If you're interested to learn more about how we do QA at Atlassian and if you're in Austin, TX on 9th of December – join us for meetup and drinks. We're also hiring, so check out this page if you're interested.
Post written by Kostya Marchenko, QA Team Lead for Developer Tools at Atlassian
