The Quality Bond in DevOps
We’re all talking about DevOps as a big movement that is changing some of the paradigms of software delivery. DevOps is the key to accelerate efficiency in software. And within the DevOps ecosystem, testing teams have a larger responsibility. As developers write several pieces of code ‘continuously’, QA teams would need to test these pieces ‘continuously’ by running regressions. Yes, even on weekends sometimes if you are releasing a hot product (like an iPhone 8 for example) to integrate it with the builds. This process, as most of you know is continuous delivery and integration. Automation and agile frameworks that drive automation obviously have a huge role to play to bring additional efficiencies to this process.
Quality, Intelligence, and the Chasm
While I ponder on this some more, what comes to mind is a practice that can make this QA process more intelligent, quick, and impactful. An intelligence that can be used not just by QA teams but by product, business, and larger engineering teams. There’s a lot of thought leadership and research in this area now and Atlassian has a great blog around Agile metrics that matter.
Both with manual and automated QA, the much-needed insight and analytics that can streamline all actions by testers is a boon for bigger and better DevOps. So how will this intelligence look like? Yes, like reports that will land straight to your inbox. So every morning, the tester, the business analyst, the engineer and the product manager just know what’s going on and what they’d need to act-on to fix things.
But Which Metrics Should You Care About Really?
This starts from understanding where the biggest gaps would be? First, from what we’ve seen, they are usually centered around driving large scale automation initiatives. The primary gap we saw at QMetry was in consolidating results from various automation tools and systems such as Selenium, Cucumber, TestNG, Maven, and others. You might be using one of these or a combination right now. However, it would be great to get a consolidated action report by churning data from all these tools. Tell us if you are thinking differently.
Traceability and Root Cause
Second, how about a complete traceability and root cause report to figure the red flags, especially crucial for agile projects with multiple parallel builds. Usually, test teams do ‘fixes’ to emerging bugs in one sprint, but the same ones sneak into another sprint. So, a mind map of sorts to show what the code was thinking and where it was going wrong would be cool to have in the bag. There’s a great blog on this which you can read here.
There are several other contenders for the third, like test coverage and defect reports, burn down charts and a few others that are usually tracked. But the last one we want to emphasize is business reports that will be consumable by all business teams. Metrics to focus on here would be global issues reported by multiple customers, trend analytics reports that can demonstrate patterns, reports with global downtimes and operational disruptions due to specific product features are seen across the board would be instrumental.
We’ve done a lot of groundwork on test intelligence for enterprise and small business teams. We will be showcasing some of this innovation at Atlassian Summit 2017 in San Jose. Also, check our products out in the Atlassian Marketplace to see where we’re going with intelligence in quality.