Blog

Using back-to-back testing in the telecom industry

First, let’s have a “helicopter view” on what comparative test provides and which cases are appropriate for back-to-back in Telecom industry. Most commonly, comparative testing is used in the case of the full replacement of OSS/BSS solution, which is absolutely crucial for any Telecom business. Rarely, new version of existing OSS/BSS is installing and requires verification.
4 November 2014
Quality assurance
Article by Julia Ilyushenkova
Head of Telco and Web applications testing department

First, let’s have a “helicopter view” on what comparative test provides and which cases are appropriate for back-to-back in Telecom industry.

Most commonly, comparative testing is used in the case of the full replacement of OSS/BSS solution, which is absolutely crucial for any Telecom business. Rarely, new version of existing OSS/BSS is installing and requires verification.

Comparative testing checks two identical OSS/BSS with the same input data in order to reveal incorrect data processing.

Three main goals to use comparative tests

  1. Detection of functional defects in Rating and Billing systems;
  2. Detection of migration errors when user data transferred from one system to another;
  3. Configuration defects identification (setting tariffs in both systems).

The general workflow for comparative test (proposed by the author while implementing projects in Telecom) is shown at the picture below.

As you can see, two identical systems are required for the testing process: the master (“Golden”) and the test system itself.

Sometimes the input data for “Golden” system cannot be accepted by the test system without additional pre-processing. Then, test data must undergo further conversion phase for compatibility with the test system. After getting results from both systems, they have to be compared – the essence of comparative test – that is, as a rule, an automated process implemented independently from environments. The final step is an empirical data analysis activity that is performed by the tester.

When we analyze the efficiency of comparative test regarding OSS/BSS solutions on real Telecom projects, we must have a critical view and list some of the shortcomings:

  • The test involves a lot of empirical work, which is difficult to automate (final stages of divergence analysis are performed manually).
  • During late iterations of the test, a lot of records with numerical discrepancy that cannot be considered as defects can be revealed. These records must be filtered from comparison
  • Sometimes the test indicates good results while in reality the situation is bad. For instance, during the collection of final statistics, two critical defects might neutralize each other when one of them increases and the other decreases the final amount.
  • Completely relevant comparative test should be performed for at least one full billing period (about one month), which is not usually the case as the test lasts less than 30 days.

As for the billing systems comparative tests requires achieving a sufficiently high level of coincidence of the output results for both systems (for instance, 99.99%). Achieving this result is a bit tricky, due to the following factors:

  • Different rounding of floating-point data in two systems;
  • Incorrect development of mapping tables (service A in the system 1 corresponds to service B in the system 2);
  • Different order of records processing in two systems may lead to divergence in tariff discounts application;
  • Peculiarity of some data types in database (example – rating results for call forwarding);
  • Dynamism of the master (“Golden”) environment. In the real life, clients are frequently changing tariff plans, phone numbers, SIM-cards, contract statuses etc. It’s almost impossible to synchronize all these changes in the test system;
  • Confirmed changes in business-logic, that is the differences in behavior between the two systems envisaged by Telecom operator requirements to the new system;
  • Uncertainty and indistinctness of the system requirements;
  • Inability to implement the requirements of the system under test.

All these factors lead to numerical discrepancies in the results, which may be up to 25% from reference values.

Nevertheless, based on my experience I would advise implementing back-to-back testing due to the many advantages that this method provides:

  • High level of test coverage for the system;
  • The opportunity to get additional metrics of the system quality;
  • Availability of implementing migration and configuration tests, not only functional tests;
  • High automation level of the testing process.

This is confirmed by the picture below illustrating results of several testing iterations on a real Telecom project.

These statistics contains three metrics:

  1. Defects – the number of functional, integration and configuration defects in the test system
  2. Discrepancy in records, % – the percentage of data units which gave different output results, relative to the originally loaded amount
  3. Discrepancy in amounts, % – the ratio of the total discrepancies amount between the master and the test system relative to the total amount of charges in the master system

Conclusion

We can note high efficiency of this type of test in relation to OSS/BSS. A good practice is to use a back-to-back test together with other traditional strategies for testing, since they are not mutually exclusive and able to detect defects of different classes of the same functionality. You should also keep in mind that an adequate strategy must be developed for back-to-back tests, taking into account all advantages and shortcomings presented in the article.

The article was published at SoftwareTestingMagazine.

More Posts

QA for financial applications: 4 reasons why it is a must-have
17 November 2024,
by a1qa
5 min read
QA for financial applications: 5 reasons why it is a must-have
Striving to release high-quality financial apps? Read on and discover 5 core reasons why your eBanking and financial solutions need software testing.
QA consulting
Quality assurance
Test automation
Why do bugs get missed
27 September 2024,
by a1qa
7 min read
Why do bugs get missed? Learn the problems and tips to avoid them
Still, finding overlooked bugs after the app goes live? Let’s find out why this happens and how to fix it.
Performance testing
QA consulting
Quality assurance
Test automation
Soft skills 101
13 September 2024,
by a1qa
4 min read
Soft skills 101: how to supercharge business success in 2025
Discover the value of soft skills for both career development and workplace improvement and learn some tips to sharpen them.
General
Quality assurance
QA to ensure smooth migration to the cloud
15 August 2024,
by a1qa
3 min read
QA to ensure smooth migration to the cloud
Learn how effectively migrate to the cloud by implementing QA activities.
Functional testing
General
Migration testing
Performance testing
Quality assurance
Test automation
Load testing
23 July 2024,
by a1qa
3 min read
7 reasons why businesses need load testing 
Want to optimize software performance or ensure its smooth functioning during peak sales season? Discover how load testing may help.
Quality assurance
Test automation
27 June 2024,
by a1qa
3 min read
Establishing seamless interaction between development and QA teams to boost productivity
Establishing seamless interaction between development and QA teams to boost productivity
Agile
General
Quality assurance
Test automation
17 June 2024,
by a1qa
5 min read
Shifting to test automation to maximize software quality
Explore in the article why businesses should move from manual testing to test automation.
Quality assurance
Test automation
RPA in QA
28 May 2024,
by a1qa
4 min read
Embracing robotic process automation to drive efficiency in QA
Discover how the convergence of robotic process automation helps reshape software testing practices.
General
Quality assurance
Test automation
QA for fintech
7 May 2024,
by a1qa
5 min read
Navigating the fintech frontier in 2024: QA’s role in delivering high-quality financial software 
Unveil the future of fintech innovations and learn to refine their quality with the help of software testing.
Blockchain app testing
Cybersecurity testing
Quality assurance

Get in touch

Please fill in the required field.
Email address seems invalid.
Please fill in the required field.
We use cookies on our website to improve its functionality and to enhance your user experience. We also use cookies for analytics. If you continue to browse this website, we will assume you agree that we can place cookies on your device. For more details, please read our Privacy and Cookies Policy.