…expresses its sincere thanks to the a1qa team for the quality assurance services provided in the course of the billing solution transformation project. Timely testing made it possible to detect and fix defects in the new solution, while test automation accelerated the validation of new releases and greatly sped up time to market.
Director of Infrastructure Projects
The client is a telecommunications company providing cellular and local telephone service, broadband Internet access, and cable television services.
The customer was creating a single billing platform to replace standalone billing systems used by the branches and subsidiaries and applied to a1qa to ensure the quality of a new billing solution. This software product covers dozens of business processes such as customer care, payments, and billing.
Successful data migration was another core objective in terms of a1qa involvement.
The client launched a unique large-scale project to upgrade the billing architecture into a single solution for all branches.
By the time the project started, each of the eight client’s branches had a separate billing system with its own characteristics and product lines.
As a result, customers connected to one branch could not receive a full-fledged service by contacting another customer’s branch.
In order to meet the client’s business objectives, the expert team developed a turnkey QA approach.
Considering the overall complexity of the solution, a1qa specifically highlighted the significance of a safe and complete data migration process.
Therefore, rigorous attention was paid to ensuring the quality of this core aspect.
The client pitched on a1qa due to the following manner of testing:
The prime task was to check the compliance of the product with the initial requirements within the integrated environment and the configured products.
To begin with, the a1qa engineers were to build a testing methodology. Therefore, the team:
The test methodology was recorded in Confluence.
To perform testing, UAT verifications were chosen as the basis. They were expanded using alternative options for completing checks, negative scenarios and tests that were not related to business processes.
All tests contained the following fields:
The test model was gradually expanding. The a1qa engineers were updating the test suite in line with further improvement of current business processes and use cases.
System testing by a1qa helped achieve the following benefits:
Throughout the whole development cycle, the a1qa engineers ensured the quality of builds by conducting 2-3 iterations of testing, which included:
Based on the results of the work carried out, the client received a document describing the testing methodology and the structure of test scenarios.
At the pre-release stage of development, it was necessary to automate testing within the integrated environment on the operator side.
Test automation objectives:
Python, Robot Framework, and REST API were applied to achieve this aim.
To develop an effective testing solution, a1qa formed cross-functional teams combining the system testing engineers and test automation specialists.
What was the trigger for this decision? When working separately, the testing teams encountered a range of difficulties:
Since the tests covered the chains of systems involved in the business scenario execution, test steps were repeated oftentimes.
To avoid code duplication and simplify test support in the future, the keywords-driven approach was applied.
Manual tests described them. The base of the defined keywords was stored in TestRail. Their coding and further reusing in tests took place during automation.
The client was performing load testing of the newly developed solution.
Its creation involved the integration of a geographically distributed infrastructure into a single system.
It was vital for the system to withstand the load that previously was distributed between 25 systems.
The client wanted to make sure the load testing would bring informative results and asked to conduct load testing methodology audit.
a1qa assigned a lead performance testing engineer to run it.
During the audit, it was required to identify the weak points of the testing processes, check the results and evaluate them for compliance with the client’s key performance indicators (KPI), as well as offer recommendations for improvement.
The audit included checking the program code, test methodology, the system for monitoring testing and obtained results.
The a1qa engineer:
All the results were successfully presented to the client’s Board of Directors.
After 4 months of rigorous audit, the a1qa dedicated expert defined more than 100 bottlenecks, suggested recommendations for improvement, and assessed their effectiveness.
The transformation of the billing solution always requires the migration of a large data amount from the source system to the target one.
Migration should not affect the routine actions of subscribers.
Therefore, the tariffs and terms of service should remain at the same level. Even a slight increase in bills or calculation errors will cause user dissatisfaction and the growing number of customer support calls.
To fulfill the task successfully, QA consultants have developed a migration testing strategy using Parallel and Dry Run methods.
Parallel Run is an approach, which allows 2 billing systems to work using the same input data while the output results are compared and the differences are analyzed.
The properly operating source system is the standard. The system that is only planned to be introduced – the target one – is compared with this standard.
The expected result presupposes that the same actions of the clients migrated are of equal effect. The effect, in this case, involves a similar amount of charges for the same services, the equal display of charges and payments on the balance, billing and fee calculation. All the traced discrepancies are potential defects in product configuration, migration, or functionality.
How is Parallel Run fulfilled? The process consists of two stages: preliminary and regular.
The objectives of the preliminary stage are:
The regular stage included identifying and fixing the issues regarding:
At the preliminary stage, only a list of selected clients that will migrate is used for comparison, while at the regular stage – all clients.
Dry Run is the process of data preparation for the Parallel Run, during which part of the subscriber base of clients to migrate is selected.
For example, at the initial project stage, clients with debts on the balance may be restricted to migrate until the debts are fully paid off.
After testing, a discrepancy analysis is carried out. All the detected defects are registered in the bug-tracking system.
Further on, the specialists gather statistics of discrepancies in terms of the verified business processes, describe the effect on the overall result of the most critical defects, and provide a detailed final report.
The advantage of Parallel Run is the extended coverage of the subscriber base and the configuration of products.
The client was satisfied with the way the a1qa team established and streamlined QA and testing processes of the new solution and ensured faultless data migration to the revamped billing.
