In mid-January 2016 the Nest ‘smart’ thermostat (owned by Google) left users in COLD. A software snag was responsible for it. It forced the battery of the devices to drain and thus it was not able to control the temperature. Nest said that the firmware update in December caused this issue along with old air filters and incompatible boilers.
Such issues can cause major support challenges and unhappy customers. It dents company’s reputation as well.
The software glitches are mostly associated with inadequate test coverage or not interpreting test coverage results right. Often, too few test cases are written for each user story while not having continuous visibility on the code on what changed during the process.
A constant coordination between development and QA team can help in better understanding the features being delivered in a release and creating test cases that cover every aspect of the feature like how it should work, constraints and validations, dependencies and ancillary functions. Agile developers build features based on a series of user stories (sometimes grouped by themes and epics). Creating your test scenarios at the user story level gives you the best chance of achieving optimal test coverage.
Test Management Tool like QMetry Test Management For JIRA can play a major role here. As it is built within Atlassian JIRA; the coordination between development team and QA team becomes very easy. Through the test coverage report provided within QMetry; test teams can dive down deep and find missing links in test scenarios.
Test Coverage Report – In Depth[/caption]
The Coverage Method:
Once your QA and development teams agree on the features to be delivered, you can begin creating tests for each feature. Coding and test development should be done in parallel to ensure the team is ready to test each feature as soon as it’s published to QA. As mentioned above; QMetry Test Management For Jira is the GOTO tool for having complete visibility on test coverage. In my next blog; I will also discuss about Traceability. Coupled with test coverage; it provides a strong metrics for having high quality code.
Some of the key tests suggested are:
Positive Tests: Ensuring that feature is working as intended with all functionalities. The error messages and workflow is working as per giving sequence.
Negative Tests: Frequently or mostly users start using the software with little or no training. Manuals are not referred as well. This leads to errors and mistakes that are unintended and not considered ever. Examples like entering special characters in numeric field or entering values that are constrained for a field are common. Missing out mandatory fields or delete records that have established relationships with other records are also reported. When you design tests, it’s important to understand the constraints and validations for each requirement and create enough negative tests to ensure that each constraint and validation is tested thoroughly. The ultimate goal is to have a ‘dummy proof’ code.
Performance Tests: Testing features under duress is very important. This is a hybrid exercise with some test cases being automated while some being manual. Testing the code with large validation dataset is the key.
Ancillary Tests: Better designed system write errors to log files, maintains change audits to records and maintain referential integrity to have clean clearing of records where master/child records are deleted simultaneously from their respective tables. The data purge routines must also be tested before deployment. Finally, security features like access rights are needed to be tested to avoid unauthorized access due to bugs. Ancillary tests ensure that log files and audits are written, referential integrity is preserved, security is embraced, and purge routines cleanly remove all related data as needed.
To conclude; by following proper methodology and having a strong Test Management add-on like QMetry Test Management for Jira; achieving proper test coverage and traceability (we will talk about it in next blog); we can build a bug-free code in agile projects.