JIRA is a great tool for managing issues and for simple test requirements; however, using JIRA for test case management can prove difficult for complex testing requirements. While JIRA relatively flat database structure offers a clear and easy to use interface it doesn’t handle the complexity needed by QA groups that need to create test cases for multiple different test scenarios, or validate functionality and UI against multiple different test environments. Also a development issue once created has a linear progression. It is opened, in progress, and then closed (hopefully). The issue might get reopened, but not on a regular basis. A test case on the other hand is most often used repeatedly. First to validate a new feature, and later to validate the existing feature still works. It may also be updated as functionality is added or enhanced while on the development side an enhancement is considered a new issue.
While at first JIRA test case management may seem like a good solution it will often bog down as the number of test cases increases. Managing a growing set of active test cases is not what JIRA was built for. QMetry’s user interface was designed to help QA teams organize and manage large number of test assets for reuse throughout the product life cycle.
QA challenges that lead QA teams choose QMetry over JIRA alone:
- The product that needs to be tested against multiple different hardware and software configurations
- The features need to be tested using different test scenarios
- QA needs to plan out their testing and provide total test time estimates for regression testing and status updates
- QA teams want full version control and the ability to verify what specific version of a test case was tested
- Management wants to be able to see what the test coverage is
- Management needs an overall better set of reports
- Developers want bugs to be reported with full environmental details.
- Organization wants to integrate manual and automated testing
With strict policies and procedures you can try to use JIRA (or a JIRA plugin) for some of the above scenarios; however, there will be limitations. And these limitations and human errors will leave gaps in the test coverage and reporting. Organizations who rely on manual test case creation from JIRA may often find that test cases are not created for many JIRA issues leaving incomplete test coverage. A simple test case may validate a features as working in one circumstance, but not for other scenarios that will be found in the field (Read more about dealing with complex test scenarios).
Capturing environmental information when filing bugs from JIRA based test cases is also a problem. Creating the bug is easy, but making sure that all the test steps and environmental factors are recorded is largely a manual task and subject to human error. Poorly recorded bugs can lead to a lot of wasted time and frustration with that circular argument we have all heard “But it works on my machine!”. (Read more about Logging Defects).
- Only import issues as test requirements based on specific search criteria (e.g. Status = Fixed)
- Automatically link test cases associated to requirements in QMetry to the original JIRA issue
- Real-time test case information from JIRA
- Timely Coverage information
- Ability to group multiple Test cases into a test scenario, what we call a Test Suite.
- Ability to quickly test and compare test suites against various platforms.
- Version Test Cases and ability to track changes across different versions.
- Support for different Automation Frameworks to integrate automated test results with manual test results for one clear
- Robust reporting Capabilities of QMetry
- Over 100 reports designed by QA teams for reporting on performance and quality to users and management
- Ability to create custom reports, with access to the underlying Database Schema of QMetry
- Professional Services available for creating Custom Reports
- Unique Coverage Status and build reports that provide powerful insights on testing
The best reason to choose integrating QMetry with JIRA is for the high efficiency and improved workflow for planning and building test coverage, and then actually running test executions. Instead of creating a test case one bug at a time, QMetry allows users to import Test Cases into QMetry from JIRA using a simple search criteria (e.g. Status = Fixed).
By importing issues from QMetry, rather than randomly creating them in JIRA, QA is able to plan out their testing cycles. It is recommended to import JIRA issues as Requirements. This allows multiple test cases to be link to or create from the JIRA issue. This workflow ensures that both functional, UI, and regression testing is covered. Using requirements also provides a much more comprehensive view of test coverage.
And although testing is done outside of JIRA QMetry use of JIRA’s API allows them to have a full and robust view of the testing status without leaving JIRA. This allows developers to see the linked test requirements and/or test case(s), with their status across the various test suites and platforms they are linked to.
In the end quality assurance is about catching human errors and you want a tool that improves quality and efficiency so that you can keep to your development schedule with a high confidence of release a quality product.
If all you need is a one to one mapping of issue to test case using JIRA (or a JIRA plugin) might work for you. If you have more complex needs then you should investigate how QMetry can provide you 100% test coverage. To find out how QMetry can help you improve quality and shorten your development cycle, click here.