What Are The Top 10 Complaints about QA? - Qmetry

What Are The Top 10 Complaints about QA?

Agile , QA , Quality Assurance , Software Testing Aug 5, 2016

Most of us that have worked with more than one development team know that the relationship between development and QA can be down right contentious.  We all want to feel part of a team and not in the   inspirational posters sense of the term (Together Everyone Achieves More).  Working together as a team applies whether you are working in a waterfall or Agile development environment. In the end QA and development have the same goal: to deliver a quality product or service.

The problem with the relationship between QA and Development is that the nature of the relationship is inherently critical.  Just as an actor doesn’t want to read their reviews a developer does not what to see the bugs filed against their work.  We aim to help you improve the communication between QA and development to increase efficient throughout the QA process.

There are common and often systemic failures in the average QA process that almost all organizations fall victim to.  Too often they are not talked about or addressed directly the problems are just swept under the rug.  Unfortunately, like a bed bug infestation, if you don’t address those bugs when they first appear they will take over your life and you will spend more time scratching the irritations than doing your work.

We’ve gathered the list below of the most common complaints from engineers about the bugs they receive from QA.

  1. Duplicate Bugs
  2. Critical Issues Found Late in Development
  3. Poorly Written Bugs
  4. Too much / Too Little Information
  5. Lack of Context
  6. Tested on Wrong Build
  7. Too Many Low Priority Issues
  8. Not a Bug
  9. Missing critical data, such as screenshots or log files
  10. Engineering cannot reproduce given the information provided
  11. Duplicate bugs

We will be dissecting each of the above complaints and offering solutions to help bring your teams closer together in separate blog articles.  Here is a quick preview of some of the highlights.

Duplicate bugs

Nothing frustrates an engineering team more than seeing the same issues over and over again, across builds, sprints and releases. There are two main fixes.  The first is to encourage your team to practice better search and discovery efforts before filing new issues. The second is to use a  test management tool that links existing bugs to test cases which can significantly cut down on the dreaded “dupes.” (Read More: How to Avoid the Problem of Duplicate Issues)

Critical Issues Found Late In Development

Nothing frustrates a team more than finding a showstopping issue in the final days of testing prior to a major release. Your QA team might have put hundreds of hours into the cycle, but only stumbled upon this issue because of how the testing was structured or ad hoc work done at the tail end.  If you are the QA Manager responsible for the team you may be left standing scratching your head “What did I do wrong?”

One common cause of critical issues being reported late in the cycle is that QA is often waiting for a stable build to start their regression testing. The simple solution often proposed for this is to have QA working alongside engineering to validate issues during the development cycle.  This is an Agile approach to QA.  The Shift Left movement is another attempt to move QA closer to engineering (e.g. QA to QE).  It is also important for QA to be able to record and report on which early development builds were tested against.  The QA team needs to be able to test aggressively, but still be able to drill down into the test results to discover potential vulnerabilities and risks introduced since the time of their initial testing.

Poorly Written Bugs

Quality extends to the content of  the bugs and steps to reproduce that you file. It’s hard for engineering to take a bug seriously if it’s riddled with spelling, grammar or context mistakes. Spell checking is one good step, but working from a template and using common terminology can help promote best communication practices for your team.

Too Much / Too Little Information

Each engineer works differently. Some engineers prefer a concise description and a screenshot. Others want the complete path the user followed in order to reproduce the issue. Know your team and agree on what level of detail works best when filing issues. It’s frequently the case that this will depend on the feature that’s being tested because developers tend to be assigned to specific features, so keep that in mind as well.

Lack Of Context

“It works on my machine” is something that QA hears often from engineering.  The best way to get out ahead of this frequent miscommunication is to provide the all-important context of the hardware, software and any other variable that have an impact on testing. It’s always best to discuss these environmental factors with engineering and agree on what is required. In this case more information rarely makes things worse.

This is practically impossible to do with spreadsheets and difficult to do with many test management systems.  A robust test management software solution can allow you to configure, execute and make sense of these complexities.

Tested On The Wrong Build

In the modern agile development cycle fixes come fast and frequently.  Development wants QA to validate features quickly so they can move on.  Continuous Integration (CI) tools such as Jenkins, CloudBees, and Bamboo help engineering speed up their build cycle, which puts stress on making sure QA is testing the right build for each feature or bug fix. Only a few test case management tools integrate with CI platforms, which can take the guesswork out of testing.

Too Many Low Priority Issues

Cosmetic issues are the low hanging fruit of testing; they are easy to spot, time consuming to report, and quickly pile up in the backlog.  The main problem is that they distract attention from higher priority issues.  Instead of developing high value features and fixes the team wades through endless discussion on low value bugs.  From a QA perspective its always important to note any feature you see, but stay focused on the larger goal at hand.

As a QA manager I always ask myself how is this impacting my user experience.  Through Agile I have learned to ask myself what is the user story?  If you can’t provide a valid user story in the form of “I am a user who wants to perform an action so that I can achieve an outcome” then maybe you shouldn’t write the bug without further discussing it with your team.  One useful feature of some testing tools is to add custom status levels such as Passed with Exception.  This allows you to communicate that there is an issue to be reviewed with the product owner without committing to a bug that may slow down development.

Not a bug

That's Not A Bug its a feature

“That’s not a bug, that’s an undocumented feature!”  Sometimes that bug you want to write REALLY IS a new feature. Being provided with the product requirements, functional spec, wire frames, user stories, etc. will go a long way to help educate your QA team about the features or planned design changes to the product.  It really helps if you have a test management tool that allows you to link test cases to test requirements that are built from or linked directly to the developer’s user stories.

This doesn’t mean every feature built to spec means it’s bug-free. It’s still critical that QA work in the mindset of an end user who may not find the design or function intuitive. If bugs are written by QA and assigned to development that demonstrate a lack of knowledge of how the product works or was intentionally designed it will drive churn throughout your process and build animosity between teams.

Missing Critical Data, Such As Screenshots or Log Files

As quickly as things change in software development and testing, one thing has remained constant: using the right tools and using them well can be the difference between success and failure. Providing your engineering team with logs to debug, screenshots with good annotation and any other supporting documentation will make their job easier.

Tired of hearing that it works on my machine?

Engineering Cannot Reproduce Given The Information Provided

Make sure the steps to reproduce are complete and accurate. Consider some of the variables you used, the environment you tested in, the platform you tested, and any other pre-conditions that led to the bug are documented. How many times have you seen a test engineer get their wires crossed in trying to communicate an issue to engineering? Many QA groups find value in peer reviewing critical issues before sending them to development in order to prevent just this kind of outcome in order to reduce churn.

Duplicate Bug

Do we really need to cover this again?!?


Dhaval Patel

Get all Latest QA Updates in Your Inbox

No sales pitches, no games, and one-click unsubscribe.

By submitting this form, you agree that you have read and understand QMetry’s Terms and Conditions.
You can opt-out of communications at any time. We respect your privacy.