Holmes Corporation was using Rally for Agile requirement gathering and needed a tool

Holmes Corporation

Holmes Transitions to Agile Testing Using QMetry Test Manager With Rally
Adopting QMetry Test Manager gives us greater confidence in the quality of a release. QMetry Test Manager’s capabilities and its integration with Rally has been a significant catalyst for the adoption of Agile testing.

Guenther Weydauer

Sr. VP Product Development Holmes Corporation

Brief

Holmes Corporation was using Rally for Agile requirement gathering and needed a tool to allow the QA to gather the requirements and submit bugs back to development.

Why QMetry Test Manager

  • Robust bi-directional integration with Rally
  • Powerful Agile testing features
  • Ability to easily gauge release readiness

Founded in 1970, Holmes Corporation is an instructional design, marketing, and channel management/distribution firm.

Guenther Weydauer is the SVP for Product Development at Holmes and is responsible for developing and maintaining over 20 different web-based products. His team follows a strict Agile process by using best practices from both Scrum and Kanban, including weekly sprints and daily standup meetings.

Holmes Corporation uses the Enterprise version of Rally Software’s cloud-based offering to manage their projects. Their developers also have developed hundreds of automated Test Cases in Selenium that are executed whenever a new product is launched or updated. However, these efforts were not yielding desired results as the QA team did not have much visibility into the testing endeavors and associated results. Further, he had to deal with scenarios where products were not tested sufficiently before Release, which resulted in significant negative perceptions from their customer base.

As the scale and complexity of Holmes’s products grew, Guenther realized that current testing processes would not suffice in the long run. His team tried to use Rally’s Quality Module for testing, but found that reusing Test Cases across Sprints was not an easy process and involved a fair degree of effort. Further, Rally’s Quality Module lacked a couple of key Reports, namely the ability to slice results by test runs against a build that would give them detailed visibility into the incremental change in stability of a release.

Guenther did a quick search for cloud-based Test Management products that integrated with Rally and decided to trial QMetry Test Manager and a couple of other products. Their other key requirements were the ability to integrate with Jenkins and Selenium and for a comprehensive Dashboard that could enable them to gauge ”˜Release readiness’.

After a month-long evaluation process, Guenther’s team chose QMetry Test Manager as the tool of choice. Apart from meeting all their needs, they found that QMetry Test Manager’s Continuous Testing capabilities enabled them to easily associate Test Cases by individual build. This feature along with the ability to move Test Cases into current scope or to the backlog, foster true Agile testing processes within the QA organization. Also, they liked the ability to pass defect information from QMetry Test Manager back to Rally and User Stories from Rally into QMetry Test Manager for Requirements.

How QMetry Test Manager helped the University of Sydney migrate from older legacy test management tool

University of Sydney

QMetry Test Manager Helps University Save More Than 50% on Test Management
We are extremely happy with pace of innovation at QMetry Test Manager. They are speedy with delivering enhancements and bug fixes via patches. If QMetry Test Manager went away it would greatly disrupt every aspect of our testing process. It is doubtful that we could achieve the same level of testing efficiencies at this cost with any other tool.

James Steward

Project Manager

University of Sydney

Brief

How QMetry Test Manager helped the University of Sydney migrate from older legacy test management tool to become more Agile in their testing practices.

Why QMetry Test Manager

  • Integration with test automation systems
  • Bi-directional integration with JIRA
  • Inexpensive to maintain

James Stewart is the Test Manager for the student administration system (SITS) at the University of Sydney. He manages a team of 15 QA engineers and is also responsible for the testing of all projects related to SITS.

When James joined the University there was an existing testing tool installed on site. This tool (hereafter “older tool”), a cumbersome, heavyweight platform from a large Fortune 100 Software Company was installed inside the University’s technology firewall. The older tool required a lot of specialized operational resources and money to maintain: hardware platforms, support personnel and licensing costs.

James wanted to develop, test and maintain a world-class student administration system at the lowest possible cost. The older tool did a reasonable job of helping to create a system with minimal defects but was prohibitively costly to operate. Further, in late-2011, the older tool version reached its “end-of-life” that is, the vendor decided to stop supporting it and required the University to undertake an expensive upgrade – it leading to an assessment of the best tool for the University’s needs.

At that time, James commenced an analysis of test management tools to find one that would address the dual goal of producing defect-free software at low costs. He decided to investigate tools that could be consumed as a service (SaaS offerings) as this would remove the need for the University to host and support a toolset and assist in reducing operational costs. After a brief online search, James narrowed the choice down to QMetry Test Manager and another tool. After a rigorous one-month evaluation process, James and his team chose QMetry Test Manager as the test management tool of choice.

Today, James and his team are using QMetry Test Manager extensively to manage their entire testing process. They are satisfied with the versatile feature set, especially the integration with defect tracking and test automation systems. With QMetry Test Manager, James estimates conservatively that the University saves more than 50% over the older tool when considering the total cost of ownership of the test management toolset. And, he has achieved his cherished goal of producing high-quality software with minimal risk of business operations.

How QMetry Test Manager helped Cognifide migrate from Testopia to handle enterprise

Cognifide

Managing Both Agile and Waterfall
QMetry Test Manager allows for testing of projects that follow both the Waterfall and Agile development methodologies. Its flexible nomenclature along with the ability to trace Requirements greatly helps us successfully manage our client interactions and for ensuring that our deliverables are on time and within cost.

Zbyszek Mockun

Head of the QA Practice

Cognifide

Brief

How QMetry Test Manager helped Cognifide migrate from Testopia to handle enterprise level testing for both Waterfall and Agile projects.

Why QMetry Test Manager

  • Need for Enterprise-Grade Test Management
  • Seamless Integration with Selenium
  • Ability to import Requirements and Defects from JIRA

Cognifide provides a wide range of digital expertise from technical design and development to digital marketing, on-going support and consultancy. With offices in London, UK and Poznan, Poland, Cognifide addressed needs of many premier corporations including Skype, Roche, Allianz and others.

Zbyszek Mockun is the Head of the QA practice at Cognifide. In this capacity he heads up a team of up to 30 QA practitioners and is also responsible for developing test strategies and the overall development of the QA group.

A few years ago, Zbyszek used Testopia for managing all their Test Cases. However, as the team began to grow he found that Testopia was unreliable and had multiple issues that precluded its use as the default test management tool of choice within the organization. The legacy tool had a number of limitations, and as a result the testing effort on larger and more complex projects was significantly higher than expected. The flexibility of the query language was limited and therefore the organization was unable to mine the data that it had captured over the months and years, rendering the tool less useful for an organization that was growing both in size and maturity.

Zbyszek decided to procure an Enterprise-grade Test Management tool that would address their needs for a centralized Testing platform that could be used internally and also for interactions with customers. The other need was for the tool to be able to integrate with Atlassian’s JIRA platform so that Requirements and Defects in JIRA could be meaningfully correlated with Test Cases. It was very important to have the possibility of integration with open source automation tools, such as Selenium. After an exhaustive search, Zbyszek and his team decided that QMetry Test Manager was the best tool for their purposes. Its intuitive UI, easy setup process and strong bi-directional integration with JIRA were key factors that influenced the team’s decision.

Today the Cognifide team uses QMetry Test Manager for testing their projects with dozens of clients. QMetry Test Manager hosts thousands of Test Cases that are accessed regularly by the Cognifide team as well as their clients.

Cognifide as an organization looks continually to push the boundaries in its ability to carry out root cause analysis in order to improve continuously. QMetry Test Manager currently meets its Requirements and Cognifide looks forward to see additional functionality developed to aid Test Management.

As the complexity of their web-based application grew Epocrates

Epocrates

Improving Efficiency of Test Management
QMetry Test Manager is the single source of truth when it comes to testing lifecycle information. Anyone can go into the tool and instantly get the information they need by drilling down into dashboard reports. Integration with Rally will further improve Requirements traceability and increase the productivity of our entire software team.

Janice Lin

QA Manager

Epocrates

Brief

As the complexity of their web-based application grew Epocrates found that Excel was too unwieldy and lacked the reporting needed for management.

Why QMetry Test Manager

  • Ability to work with Agile development methodologies
  • Strong integration with JIRA
  • Insightful dashboards and powerful reporting

Epocrates, Inc. creates point-of-care digital solutions that enhance the practice of medicine. More than 1 million active members, including 50% of U.S. physicians, use Epocrates to help improve patient care and practice efficiencies with its drug reference, educational and clinical apps.

Janice Lin is a QA Manager in the technology organization at Epocrates. She is a key influencer in evaluating and selecting technologies for use within the QA organization. For years, her team was using Excel to create and maintain test cases. However, as the complexity of their web-based application grew, Janice found that Excel was getting too unwieldy and unmanageable. They also needed to instantly get access to various summarized information and analysis about the state of their testing cycle.

Janice decided to invest in a Test Management tool that would help centralize all Testing activity and also provide all the desired results to both her executive management and QA teams. Their primary requirement was that the selected tool would be able to integrate with their defect tracking tool, JIRA that was being used extensively by their engineering team. They also wanted a tool that had an intuitive UI and powerful reports that could help them get the information they wanted at the desired level of granularity.

Janice and her team evaluated 4 vendors including QMetry Test Manager for use within the organization. They finally chose QMetry Test Manager as it met their main criteria for strong integration with JIRA and for the ability of their dashboards to provide the analysis and reports they needed. Further, they were pleased with the level of support they received from both the sales and support teams for installation inside their corporate firewall and for getting acclimated with the numerous features inside the platform.

Today, QMetry Test Manager is regularly used by many groups inside Epocrates including QA and product management. Plans are in place to integrate it with their requirements management platform, Rally, to introduce new efficiencies into their entire ALM processes.

As the QA team grew and time to market pressures increased

j2 Global

Moving From Excel to a Central Collaborative Test Case Repository
The QMetry Test Manager tool and its easy to use UI have been very helpful for us. QMetry Test Manager has helped us to manage the collaboration of test cases and made it easy to assign and run test cases.

TJ Dong

Senior Software QA Lead j2Global

Brief

As the QA team grew and time to market pressures increased, the Excel based test management process became cumbersome with several inefficiencies.

Why QMetry Test Manager

  • Intuitive user interface
  • Bi-directional integration with JIRA
  • Centralized repository of test assets

j2 Global Communications provides a variety of business-critical communications and storage messaging services through its global communications/telephony/messaging network. j2 Global Communications offers a complete array of unified-messaging and communications services, including faxing and voicemail solutions, web-initiated conference calling, storage messaging services.

j2 Global’s QA team ensures quality for the company’s websites that deal with changes in billing, payment, email systems, messaging and several other mission critical projects. The team was using MS Excel in conjunction with Confluence to manage the testing process. Some of the key challenges included:

  • Collaboration: With distributed QA and development teams, there was no way to communicate and collaborate to maximize their testing efforts.
  • Availability: The client tried to store Excel sheets in a centralized location, but accessibility and availability remained a challenge.
  • Visibility: There was no way for QA managers and other team members to see the test progress in real time or to easily create comprehensive reports.
  • Different structures: Since all team members were working independently, it was very difficult to enforce standards and the test assets would often end up being in different structures.

j2 Global Communications researched various solutions available in the market that could potentially solve the problems the QA team was facing. The company had the following main considerations as selection criteria:

  • Integration with JIRA defect tracker: j2 Global was already using JIRA as their defect tracker and wanted to leverage their existing investments rather than switching to another defect tracker.
  • Usability: The Company wanted to waste no time in starting to organize its testing. Hence, the UI needed to be intuitive and should require minimum training to get up and running.
  • Adaptability: j2 Global did not want to change the workflow of applications to suit the test management tool, but rather wanted the tool to adapt to their workflow and processes.
  • Powerful test case management: The number and complexity of test cases in j2 Global’s testing environment required comprehensive test case management capabilities.
  • Cost efficiency: This would be the first test management tool implementation by the QA team at j2 Global, so the company was looking for a solution that would fit within their budget.
  • Reporting: Monitoring the testing process and progress was very important for optimization and providing stakeholders with the required visibility.
  • j2 Global evaluated several test management tools including QaTraq and ApTest among others before selecting QMetry Test Manager as their tool of choice.

j2 Global was highly impressed with QMetry Test Manager’s intuitive interface:

Another main consideration for j2 Global was the integration with their defect tracking tool JIRA. QMetry Test Manager’s bi-directional integration with JIRA provides a seamless experience to the end users, regardless of whether they are logged in from JIRA or from QMetry Test Manager.“Another feature that we really liked about QMetry Test Manager was its integration with JIRA,” said TJ, “this, along with powerful test case management, test plans and requirements management modules, addressed all the challenges we were facing and made our decision very easy.

QMetry Test Manager’s centralized repository of test assets like test cases, test plans, reports etc has ensured availability and accessibility around the clock, from anywhere. The QA team can now generate, export, email and print a large number of reports to communicate test progress to other team members. Integration with JIRA provides an effective communication channel for QA and development teams for better collaboration. QMetry Test Manager’s version control and change history enables the QA team to keep track of the changes made and all members are updated in real time.

QMetry Test Manager’s workflow and test plan management has helped j2 Global to set a standard process across the testing cycle, instead of using an ad hoc process. This has also helped them to bring consistency across the board. The Test Suite module allows testers to choose different platforms and execute multiple tests on each of the platforms.

QMetry Test Manager is now an integral part of j2 Global’s QA process and has helped them to organize their test process and optimize testing efforts.

How eXo Platform uses QMetry Test Manager integrated with Selenium automation to scale QA

Exo

Scaling QA with QMetry Test Manager and Selenium
Our team is very pleased with the functionality of QMetry Test Manager and the timely response we get from their support team whenever the need arises. QMetry Test Manager is absolutely critical to our continuous efforts to deliver state-of-the-art technology solutions to our customers.

Patrice Lamarque

Vice President of Products

eXo

Brief

How eXo Platform uses QMetry Test Manager integrated with Selenium automation to scale QA to keep up with growing market and deliver state-of-the-art technology solutions.

Why QMetry Test Manager

  • Synchronize with their cloud-based LDAP
  • Integration with automation processes in Selenium
  • Ability to test a fast-growing and complex product line

eXo Platform helps companies connect their employees, customers and developers through social, collaborative and content-driven intranets, websites and dashboards. As the VP of Products at eXo, Patrice Lamarque, is responsible for defining the product roadmap, marketing and also for Functional QA — that is, ensuring that any product release meets the needs of their diverse set of customers.

When Patrice joined the eXo team, they were using Excel for helping conduct tasks relating to Functional QA. He quickly discovered that Excel was insufficient for eXo’s fast-growing and increasingly sophisticated product, and for supporting a rapidly growing customer base. Put succinctly, Excel was just not suitable for the natural process of corporate maturation that eXo was going through.

Patrice decided to invest in an Enterprise-grade Test Management tool that would help alleviate the problems that his team was facing. His key criteria were affordability, for the solution to be based in the cloud (to avoid IT overhead), and for it to be able to integrate effectively with the in-house defect tracking tool, JIRA.

After evaluating 2 commercial and 2 open-source tools, Patrice and his team settled upon QMetry Test Manager. The product had all the capabilities they wanted and it was also at a price point that made it a very easy decision for eXo. They were particularly impressed at the ease by which Test Cases could be grouped and executed in Test Suites. Further, QMetry Test Manager ”˜ticked off all the other boxes’ in their desired criteria for a Test Management platform: the ability to integrate with Selenium, a strong bi-directional integration with JIRA, ability to synchronize with their cloud-based LDAP system, and a powerful user interface that made it very easy for a new user to get familiar with the product’s functionality in a short amount of time.

How Proteus uses QMetry Test Manager to manage testing across multiple mobile devices

Proteus

Managing Mobile Testing
Today QMetry Test Manager is an integral part of our testing processes. Without QMetry Test Manager we would have to recreate and re-engineer many of our activities, and this would significantly affect the speed with which we release applications in the marketplace.

Scott Andrews

Senior Manager of Systems Integration

Proteus

Brief

How Proteus uses QMetry Test Manager to manage testing across multiple mobile devices and tablets and report bugs through Assembla.

Why QMetry Test Manager

  • Ability to test on a large number of mobile platforms
  • Strong integration with Assembla
  • Enterprise-grade platform with intuitive UI

Proteus Digital Health is a pioneer in the digital health industry. Their sensor-based technology enables customers to better manage their health and more effectively collaborate with caregivers and clinicians, while enabling new information-based business models.

Scott Andrews is the Senior Manager of the Systems Integration group at Proteus Digital Health. Part of his responsibilities include ensuring that the ingestible sensor, all the front-end devices like mobile phones and tablets, and back-end systems like servers and applications, are designed to work in harmony. His group is also responsible for ensuring Proteus’s applications will work accurately on all supported devices and at the performance levels required by end-customers.

Scott’s group previously used Excel and trace matrices along with some capabilities in bug tracking systems to conduct test management. However, Scott quickly discovered that Excel had many inherent limitations especially as Proteus had to increasingly support a larger number of devices. He decided to invest in an enterprise-grade test management platform that would enable his team to conduct effective test management for a variety of platforms. Other key needs were for the product to integrate with Assembla — the defect tracking platform used by Proteus’s development team; for the product to have a good integration with Selenium and for the product to be hosted in the cloud.

Scott decided to use QMetry Enterprise as a SaaS deployment. His team found that QMetry Test Manager met their needs, was inexpensive and had an intuitive UI that could enable them to quickly create test cases and associate them with various platforms. Today, QMetry Test Manager is widely adopted amongst Scott’s team at Proteus as the test management tool of choice.

QMetry Test Manager helped Healthland move their QA department

Healthland

Managing QA in an Agile Development Processes with JIRA
Candidly, I don’t know how we would carry out testing without QMetry Test Manager. We would be in a world of hurt and would regress back to a scenario of ad-hoc and undefined testing.

Anne Frazer

Director of Quality Assurance

Healthland

Brief

QMetry Test Manager helped Healthland move their QA department to keep up with a the new Agile development processes from engineering.

Why QMetry Test Manager

  • Ability to work with Agile development methodologies
  • Strong integration with JIRA
  • Flexible and customizable

Founded in 1980, Healthland (formerly Dairyland Healthcare Solutions) has been serving the healthcare market for nearly 30 years. During that time, Healthland has grown into a leading nationwide provider of comprehensive healthcare information systems for rural and critical access hospitals.

In 2011, Healthland’s technology organization appointed Anne Frazer as its Director of Quality Assurance. Anne’s charter was to build out a Quality Assurance team, to provide strategic direction for quality incentives and establish formal processes for robust testing of Healthland’s products. Today, Anne is managing a team of more than 30 Quality Assurance professionals distributed across offices in the US and overseas.

When Anne took over the reins of the QA group, Healthland’s customers were expressing dissatisfaction with the quality and reliability of the product. The QA team itself was not happy with the capabilities of the current QA tool, SmartPlanner; and there was inconsistent QA process across offices to organize test cases and run them against chosen platforms.

Anne initiated the search for a new QA tool that would satisfy their dual needs of an affordable and feature-rich test management solution, and one that would fit in with their Agile software development methodology. The other key requirement was for the tool to integrate with their defect tracking tool, JIRA with GreenHopper plugin (GreenHopper is now known as JIRA Agile).

Anne and her team evaluated 4 solutions and decided that QMetry Test Manager was the best fit for their feature requirements and budget. The team found that QMetry Test Manager had all the relevant, industry-grade features that were required to define and execute a comprehensive testing strategy. Further, they found that it was extremely flexible and allowed them to define and organize test cases into repositories exactly the way they wanted. They were also enthused by its ability to fit into their Agile processes and the fact that the nomenclature in the tool could adapt to include Agile terminology.

Today, QMetry Test Manager is accessed very regularly by Anne’s team and is central to many quality-defining initiatives at Healthland. It has even helped them uncover latent bugs within 3 previous release cycles. Further, they are impressed by the data portability capabilities and the open API to Perl. The tool is particularly popular with her offshore teams.

SolidFire neededa lightweight and inexpensive SaaS test management tool

SolidFire

Low-Cost SaaS Integration with Fogbugz
QMetry Test Manager provides a holistic life cycle management solution and is strongly engrained in our development process. If QMetry Test Manager went away it would definitely cripple our development process and even affect our intellectual property in the short and mid-term.

Havish Vinnakota

Director of Quality Engineering

SolidFire

Brief

SolidFire needed a lightweight and inexpensive SaaS test management tool with enterprise level version control and integration with Fogbugz.

Why QMetry Test Manager

  • Bring efficiencies into testing process
  • Organize testing artifacts
  • Integration with defect tracker tool FogBugz

Havish Vinnakota is the Director of Quality Assurance (hereafter, QA) at Solidfire, a fast growing company in the storage industry. In this capacity, he works closely with the software development and program management teams to develop processes for maintaining product quality. Havish joined Solidfire in its early days and was tasked with building out a QA team and defining standardized QA processes that would help Solidfire release quality, zero-defect products.

Havish was looking for a comprehensive test management solution that would help him and his team organize all their testing artifacts. This in turn would help standardize testing processes and foster better collaboration amongst team members. Havish was looking for a tool that was lightweight and inexpensive and one that did not involve further investments in hardware and maintenance personnel at their end. His other key requirement was that the chosen product should integrate with their existing defect tracking system, Fogbugz.

Havish had used tools from large software vendors in the past, but had found the products to be bloated, to have bad usability, and an overkill for their needs. He spoke to Fog Creek Software (manufacturers of Fogbugz) and acquired a list of Total Test Management solutions that integrate with Fogbugz. Then he went to evaluate 4 products that satisfied his initial selection criteria and eventually settled on QMetry Test Manager. Apart from a robust integration with Fogbugz, Havish found that QMetry Test Manager was affordable, had a great user interface, was speedy and could be accessed from the cloud. Further, it was great to be able to connect its Requirements module to cases in Fogbugz, and to be able to run scripts automatically against an in-house automation engine.

Solidfire has now been using QMetry Test Manager for over a year now and it’s been widely accepted by the product management and QA teams. Havish’s team has grown to more than 12 members and a total of 30 Solidfire access QMetry Test Manager on a regular basis. There are more than 1000 test cases in QMetry Test Manager now.

A growing distributed QA team needed a better solution

Stanford

Migrating from Excel to an Integrated Solution with JIRA and QMetry Test Manager
QMetry Test Manager is central to our testing efforts now. It has helped me build a more professional QA organization within HighWire. My team is significantly more productive than the days where we used Excel for Test Case Management. I definitely think it can be used by my counterparts in other departments at Stanford.

Vusal Azizaliyev

QA Manager

Stanford University

Brief

A growing distributed QA team needed a better solution to collaborate between themselves on test cases and with development using JIRA.

Why QMetry Test Manager

  • Bi-directional integration with JIRA
  • Easy visibility into testing content
  • Bring efficiencies into testing process

Located in Palo Alto, California, HighWire is a division of the Stanford University Libraries. As the leading ePublishing platform, HighWire Press partners with independent scholarly publishers, societies, associations, and university presses to facilitate the digital dissemination of 2,500 journals, reference works, books, and proceedings.

The Quality Assurance (QA) team at HighWire consists of 17 engineers located both on-site and offshore and is responsible for both quality control and quality assurance processes for all HighWire products and releases. Just less than 6 month ago, the HighWire QA team was managing test cases in Excel with no particular reporting capabilities, inability to view key test case data through graphs and no formal process or software for managing test cases. A newly appointed QA Manager decided to change this and initiated research on the capabilities of leading test management platforms.

The key requirement for a test management platform was a strong integration with the already adopted issue management platform, JIRA. After an initial search, the QA team narrowed down the search to 3 platforms – QMetry Test Manager, a free test management tool and a larger testing application from one of the largest software vendors in the industry. QMetry Test Manager was the best fit. Neither was it expensive and heavy as the offering from the large vendor nor was it as bare-bones as the free software. Also, some members of HighWire’s offshore QA team had used QMetry Test Manager in the past and they were comfortable with its UI and functionality. Further, QA team found the Sales and Support teams at QMetry Test Manager to be extremely responsive to their needs.

Today QMetry Test Manager is being used very actively at HighWire. The team is particularly pleased with the insightful charting and graphing capabilities as well as the ability to organize and execute test cases within test suites. The bi-directional integration with JIRA is also immensely helpful as it enables a meaningful linking of test cases and requirements with defects and for providing a centralized place to add comments. Further, the QA team at Highwire has connected their automation framework through QMetry Test Manager’s Web Services API in a way that enables them to view test results in QMetry Test Manager.