Portfolio
Helping a leading Japanese enterprise in delivering high-quality hardware solutions
The client engaged a1qa for professional QA assistance to ensure the flawless operation of a retail hardware solution.
Overview
Within a1qa’s responsiveness and complete dedication to the project, we admit that the company is a reliable software testing provider able to rapidly address urgent issues and perform all the activities within a pre-defined budget and timelines.
Managing Director and Finance Director at a software development company
The client is the No.1 IT service provider in Japan and included on Fortune’s “World’s Most Admired Companies” list. They focus on B2C and B2B markets in 180 countries across multiple industries. The client engaged a1qa to ensure faultless operation and accelerate QA for their retail hardware solution, an MVP-based self-checkout simulator, through test automation.
Services offered
Test automation
QA workshop
Project scope
The client, which produces software and hardware systems for large-scale entities such as plants, shopping malls, logistics centers, and more turned to a1qa to ensure the quality of their retail hardware solution — a self-checkout station.
Testing on hardware was not cost-effective. Therefore, the customer developed an MVP-based self-checkout simulator that could be deployed both as a desktop app and a web application.
To facilitate QA activities, the IT solution mimics the interaction between hardware sensors and controllers and the installed software on the station, leveraging test data, including bar codes, weight values, and payment methods.
The customer needed to focus on core activities and speed up regression checks with automated testing. As their in-house QA engineers lacked the appropriate experience, the client engaged a dedicated QA team from a1qa.
A pilot project
To demonstrate expertise in test automation, a1qa conducted a pilot project and established test automation processes and approach from scratch, tailored to the customers technical stack and infrastructure standards.
From the start of the project, a1qa integrated 1 QA automation engineer and 1 QA automation manager into the client’s processes, infrastructure, and in-house QA team.
- Configuring test environments
Before testing, the a1qa team needed to set the test environments and ensure correct test data usage.
As the solution under test consisted of a great number of interconnected components and subsystems, QA specialists were constantly communicating with the product team to configure the environment as soon as possible and prevent issues with QA activities.
- Performing test automation
To automate testing, a1qa specialists developed C#-based scripts for the web version of the self-checkout simulator after receiving the test cases scope composed by the client’s QA team. The pilot project resulted in the creation of five automated tests that detected critical defects that the manual QA team failed to detect. By thoroughly analyzing the root cause of the faults, the a1qa team determined that bugs were reproduced within fast user flows. Thus, test automation helped mimic real-life end-user actions while providing the opportunity to run tests on various speed modes of user flows.
- Tailored test automation solution
Selenium WebDriver, an industry-standard library for interacting with a web interface of the self-checkout simulator, was chosen by the a1qa team after working with a wide range of test automation tools.
The a1qa specialists identified the potential risks of using Selenium WebDriver. When interacting with the tool’s code directly, any minor change to its interface may cause expensive and time-consuming amendments to all the written scenarios.
To eliminate such risks and optimize QA workflows, QA automation engineers built a three-layer solution:
1. A base layer — Selenium WebDriver — to interact with the web solution, for instance, click buttons, download files, and enter text.
2. An intermediate layer — a1qa’s proprietary Aquality Automation framework — to interact with Selenium WebDriver and pull up the necessary information for a core automated script. The team designed this layer to enable prompt changes within any modification to a solution or library. Additionally, it simplified test data usage by converting it from CSV/JSON to XML, which is used by the self-checkout simulator.
3. A top layer — C#-based automated scenarios. The a1qa professionals enabled the performance of software testing on short notice using this layer, while also allowing for the creation of test automation scenarios based on the core script
The a1qa team automated script changes, prevented test duplicates, and mitigated the risks of large-scale automated tests alterations.
To consolidate test artifacts, the specialists recommended migrating test documentation, bug reports, and source code of the test automation approach into the Azure DevOps space.
Organizing the test automation process, the QA engineering team established a final resolution while configuring the CI pipelines and reporting dashboards in Azure.
QA workshop
In addition to performing test automation activities, the a1qa team conducted a workshop to demonstrate to the client’s QA specialists how a specifically designed test automation solution operated and what business benefits it brought to the table.
The two-hour workshop consisted of three sections:
Section #1. The tech part of the solution
To provide step-by-step instructions for the manual QA team, the a1qa engineers executed the entire automated tests life cycle in real-time. Test automation specialists showed how they set up the test environment, organized the framework, its structure and architecture as well as used test automation design patterns for regulating the speed of test execution.
What’s more, they demonstrated how Azure DevOps reporting plugins facilitated obtaining test results within bugs detected during test automation.
With complete automation guidelines, the product team was able to execute the automated tests on their own, explore the results, and measure the quality of the simulator release from the regression perspective.
Section #2. The business advantages of the solution
The a1qa team compared the time spent on manual and automated testing. That resulted in 4 hours and 7 minutes correspondently for 5 test cases. Therefore, QA engineers showed how the solution speeds up the delivery time while spotting hidden bugs promptly and saving manual efforts.
Section #3. Results
Upon approval from both technical experts and management, the client received the deliverables: test automation scripts, a continuously supported framework by a1qa’s test automation community, issues in the bug-tracking system, configured infrastructure and execution pipelines, and a detailed guide for automated testing.
Further partnership
Following the pilot project’s success, the client expanded with a1qa adding them to a vendor list and extending the scope of test cases to 136 automated scenarios encompassing all possible user flows.
To optimize the test automation process and free up the client’s manual QA engineers, the a1qa team established a CI system from scratch by integrating the three-layer automated testing solution with Azure DevOps.
Additionally, a1qa migrated the documentation process to Azure DevOps, creating a more unified approach to handling test artifacts.
As a result, the entire regression testing flow was reduced from 45 hours to 2 hours. What’s more, any project member was able to execute tests, detect software bugs, get checks results, and document the software quality level without the presence of the test automation specialists.
By completely dedicating to the project, rapidly addressing urgent issues, and performing all activities within a pre-defined budget and deadline, a1qa helped the client accelerate regression testing, free up QA resources for core tasks, and ensure that the hardware worked flawlessly.
Technologies & tools
- Selenium WebDriver
- Aquality Automation
- Azure DevOps
- C#
Results
-
Helped speed up time to market: the entire regression testing flow was reduced from 45 hours to 2 hours.
-
Set up a CI system encompassing the entire test automation process while freeing manual QA engineers for core tasks.
-
Detected hidden software defects due to simulating fast users’ flows.
In numbers
6
months of cooperation
2
dedicated test automation specialists
136
automated scenarios created
22.5X
reduction in regression testing time