Do not treat crowdtesting as panacea. Interview with Parimala Hariprasad. Part II

In the previous post we talked to Parimala about Mobile testing and UX testing. In case you missed this part of the interview you can read it here. This time we talk about crowdtesting and its pros and cons.

a1qa: You recently moved to an organization which offers Crowdtesting services in addition to Offshore Testing Services. Can you talk about Crowdtesting and also explain how Managed Crowdtesting different from regular Crowdtesting?

Parimala Hariprasad: Crowdtesting or Crowdsourced testing is an on-demand software testing service, delivered through highly skilled & qualified, geographically distributed professionals over a secure private platform. In a way, this can be considered  a type of software testing outsourcing, where you delegate testing responsibilities not to a single vendor, but to a range of them simultaneously.

Managed Crowdtesting
A qualified project manager, who is typically a proven community leader, designs or reviews test strategy, and approves or amends them to cater to customer’s specific testing requirements. Each project includes an explanation and access to a forum where bugs and issues are discussed and additional questions can be asked. Testers submit documented bug reports and are rated based on the quality of their reports. The amount the testers earn increases as their rating increases. The community combines aspects of collaboration and competition, as members work to finding solutions to the stated problem.
a1qa: What do you think about using Crowdtesting as an augmented approach to testing?

Parimala Hariprasad: Every organization maintains certain ethos with comfortable work culture, talent pool of people and scores of technical debts. Software testing is the least important areas to spend money on for many organizations, be it 50 years ago, today or even 50 years later. Because, according to customer, software testing consumes money, it doesn’t bring money. In such a situation, selling software testing solutions bundled in different packages to customers as “the most innovative solution of the century” no longer makes sense.

The need of the hour solutions is to pitch testing models/solutions as an augmented approach to testing.

Scenario 1 – An organization which employs traditional testing methodologies approaches you for testing.
An organization, let’s say, has a mature testing process in place and also has a Test Center of Excellence. How to add value here? It’s important to understand the needs of the customer, identify the pain points they are going through as a result of not testing or doing poor testing and pitch a model that fits best for them. Customer might take a couple of test cycles to gauge if that model works well or not.
In this case, if bringing a fresh pair of eyes or several of those helps, then suggesting several new team members in the organization/team to initiate testing helps. If it needs to be done on a larger scale, Crowdtesting can be an option. Note that it is not the only option, but one of the options.

Scenario 2 – An organization is looking for diversity in test configurations and devices.
Consider a large organization where web or mobile applications are accessed from different operating systems, browsers and browser versions, multiple mobile devices, different platforms like Android, iOS, Windows OS, several manufactures, different screen sizes and resolutions. From a cost and time perspective, organizations often find it hard to test on a variety of test configurations. Such a context is suitable for crowdtesting where a professional testing community works in a Bring Your Own Device (BYOD) model and test the application, hence giving broader device/platform coverage.

Scenario 3 – An organization wants to solve its regression testing problem.
Many legacy applications have a need for regression testing. While new features are conceptualized and implemented, the pain of maintaining existing features from breaking is a big pain. This risk is further aggravated given the number of operating systems, browsers, mobile devices and other test configurations. Regression testing candidates are a great fit for Crowdtesting where the crowd is capable of regressing on a variety of platforms and test configurations within a short period of time.

a1qa: What are the advantages and disadvantages of Crowdtesting?

Parimala Hariprasad: Crowdtesting has its advantages and disadvantages. It is up to organizations and their customers on how they can use it and derive value out of it.

Advantages

• Representative scenarios from the real user base
• Tight feed-back loop with rapid feedback processing and agility
• Comprehensiveness in use cases, platforms, tools, browsers, testers, etc. that is very hard to replicate
• Cost efficiency
• Diversity among the pool of testers lends to extensive testing
• Reduced time to test, time to market and total cost of ownership as most defects can be identified in relatively short time, which leads to significant reductions in maintenance costs

Disadvantages

• Governance efforts around security, exposure and confidentiality when offering a community project to wide user base for testing
• Project management challenges that stem from the testers’ diverse backgrounds, languages and experience levels
• Quality assurance efforts to verify and improve bug reports, identify and eliminate bug duplicates and false alarms
• Equity and equality constraints in the reward mechanism with remuneration as a function of the quality of contributions that meets a prescribed minimum standard

a1qa: What does the future hold for crowdtesting?

Parimala Hariprasad: Crowdsourced testing, clearly, has its advantages and limitations. It cannot be considered as a panacea for all testing requirements and the power of the crowd should be diligently employed. The key to great Crowdtesting would be to use it prudently depending on the tactical and strategic needs of the organization that seeks Crowdsourced testing services. It is important for the organization to embrace the correct model, identify the target applications, implement Crowdtesting, explore them for few test cycles, monitor test results and custom design the model to suit their needs.

a1qa: And finally, what does Software Quality mean to you?

Parimala Hariprasad: I follow Jerry Weinberg’s definition for Quality – “Quality is value to some person” which was further improvised as “Quality is value to some person who matters” by James Bach and Michael Bolton.

I was recently talking to a friend who wanted users to install his mobile app and travel from point A to point B in 10 different cities across the world. The first thought of it generates laughs and gags for its weird requirement. However, for this guy, getting this done means the world to him as it is helping him analyze how his product is behaving in contexts and weather conditions amidst other environmental/technical factors. For this customer, value is to actually see the value of his product work or not work so he can make necessary amends.

Value, in my opinion, is defined by the customer based on the

1. Pain points customer is facing
2. How much customer is willing to pay to remove those pain points?

As we move farther and ahead in technology, it is becoming critical to pay attention to the value the customer is looking for. Offering good value means offering good quality. This applies not just to software testing, but to any craft where “Value” is valued!

“Quality is about How People Feel”

Parimala thanks for sharing your experience and thoughts.We`ll be glad to talk to you again.

You can read Parimala blog.