The client is one of the leading US-based manufacturers of home appliances operating globally and possessing several trademarks offering an extensive blend of products.
The customer’s software under test includes B2B and B2C dimensions.
B2B solutions embrace:
B2C software encompasses:
Aiming to reduce hosting costs and improve overall organizational performance, the client was undergoing migration to SAP Hybris Commerce Platform. To assure its quality by thorough UAT, introduce test automation best practices, build performance testing from scratch, and shape QA processes, the client partnered with a1qa.
a1qa started by delving into the technical core of the solutions, which is based on the seamless interaction of the back- and front-ends.
Front-end involves:
Back-end incorporates:
Having learned all products’ nuances, the engineers got down to the quality assurance scope.
Functional testing was applied to the SAP Commerce Cloud customizable features:
UAT incorporated verifications to make sure the whole functionality scope was operating as it was intended, and the correct information was passed among various system components. Test cases representing real user scenarios included detailed steps from opening the website to buying a certain product.
Testing scenarios that covered SAP ECC had deeper test coverage to verify vital business details (accuracy while creating invoices, disposal of goods, transfer of products between warehouses, etc.).
Test automation with its primary objectives of speeding up time to market and reducing operational costs was pivotal to streamlining the testing scope.
Automated release verification tests were performed against the operation of the entire project’s portfolio. The essential component of tested functionality concerned placing orders. Many aspects could influence this process (e.g., product type, location, registration, delivery, availability in a warehouse), requiring certain preconditions for each order — which were out of the QA team’s control. To adapt to constant changes, the QA experts created the following workaround to address that challenge:
As soon as the transaction is performed, SAP ECC creates a message about the successful operation, the presence and details of which are checked by this test.
a1qa designed a special approach to prevent lost time on analyzing false positive results of automated tests. If during step 1 the product wasn’t found, the test was specifically highlighted. The QA experts could easily understand what was missing, quickly fix the problem, and rerun a test suit.
Test automation introduction was effective: in just two months, it contributed to saving 50 business days of the manual testing team consisting of 4 people. All in all, test automation by a1qa curtailed the time required for release verification testing by 90%. The QA team is constantly working on expanding test coverage.
To help the client generate high conversion rates, increase revenue, and boost customer experience, the a1qa engineers should ensure the software would operate under high loads during Black Friday, Cyber Monday, and other periods of increased influx of customers.
The client already had a team responsible for this purpose, but their Black Friday testing strategy wasn’t fully defined and didn’t consider all variations in user behavior during this period. Therefore, the experts prepared user journeys emulating realistic loads, which later became the baseline for scripts.
While creating them, a1qa enhanced the website load profile. The QA engineers took into account different user behavior models and included both the most widespread and the least common actions to verify the results of placing an order, adding products to a wish list, and scrolling through a website. The team developed around 200 performance test scripts.
Afterward, all solutions were verified before each release by harnessing the activities listed below:
Suggested performance testing improvements:
To decrease release delay risks, a1qa recommended the client fulfill performance testing during the sprints that deliver features prone to performance bottlenecks, increase sprint length, and cover not only functional testing but also the required performance testing verifications.
When a1qa joined performance testing activities, the client was using an expensive tool. To help the client optimize operational expenditure, a1qa’s engineers identified the set of features that the tool should have and proposed switching to an open-source solution with the same capabilities. After its successful usage during a pilot period of several testing rounds, the entire performance testing team started using it on a regular basis.
It was intended to guarantee seamless integration of:
Each payment platform (Paymetric, PayPal, Affirm) possesses a set of rules to manage possible issues that may occur during money transactions. After entering, credit data is sent to the system for validation. The type of response depends on the variants of the issues spotted or the situations of processing the data. The tests check that all those types are considered by the system and are validated correctly. For testing on production environment, the team harnessed only real credit cards provided by the client.
Considering complex taxation in the USA, it was vital to make sure the correct tax is assigned to a certain state. Taxes are calculated in the following way. The request for a certain tax is sent to Vertex, then the number received is transferred to an order. After this order is created in SAP ECC, the process repeats once again to ensure the maximum accuracy. Testing helped guarantee the absence of software glitches and correct calculation of taxes.
Apart from testing, a1qa contributed to the following:
Provided full release support
The client delivers new functionality once every 2-3 weeks. To ensure the quality releases and compliance with deadlines, a1qa designed an approach to training and sharing constantly expanding expertise on all subprojects so that all QA team members knew the nuances of any software module within all subprojects. Should any necessity to augment the team preparing for the release arise, the engineers are swiftly redeployed to support the process.
Ensured smart team scalability
To cover changing workload in terms of multiple subprojects and optimize QA costs, a1qa ensured a knowledge retention process, enabling a1qa to ramp teams up or down upon client’s request.
When the client started a new subproject, a1qa dedicated specialists with a vast project background from other subprojects or onboarded new members and helped them quickly delve into the client’s systems.
Aligned processes
Due to multiple subprojects, each group followed its own habitual patterns that resulted in a desynchronized testing process. To increase organizational performance, a1qa unified the standard pool of tasks carried out by the QA experts. For that, they:
Helped shift to Agile
Initially, only UAT followed Agile practices. However, having demonstrated its perks like efficient defect fixing and communication with developers, faster time to market, and clean code, a1qa contributed to shifting overall development process to Scrum.
Developed approach to assess the quality level
The work fulfilled by multiple teams was evaluated according to miscellaneous systems, which impeded the overall assessment of system quality, release readiness, and specifying the definition of done. To standardize it, a1qa set up a unified approach to an impartial evaluation of the quality level within teams.
Proactively participated in PI planning
At times, user stories weren’t detailed, leading to critical defects that occurred late in the development process. Therefore, a1qa joined the client’s regular meetings to plan the workload, provide and discuss insights on the proper system operation. A team of business analysts was also assigned to identify and clarify the requirements.
Adjusted to the client’s time zone to the fullest extent
To reach the maximum overlap in time, sync up, and solve process questions together, a1qa set some rules for all its members regarding the distribution of work activities during the day. Thus, isolated project activities not requiring other parties’ involvement were conducted during a certain period of time. Other time slots were available for cooperating with the client’s R&D teams and project stakeholders if required without the need to rearrange or postpone planned activities or work overtime.
Mitigated business risks
To eliminate the possibilities of downtime, a1qa developed a roster for all the QA teams. Continuous monitoring and defect validation were observed due to introducing 4-hour shifts.
Strengthened organizational performance
Particular efforts encompassed fixing pitfalls that occurred on the side of 3rd party vendors, which led to an increase in operational costs and testing time. Having analyzed their root causes, a1qa created a report showcasing the current state of things together with the list of improvements required. It allowed the client to reassess the efficiency of all the teams involved in product development and testing to eliminate bottlenecks and enhance project workflows further.
Problems within Accertify
Its configuration is close to the production environment. Therefore, the number of orders placed by the engineers for 1 day from 1 IP address was limited due to business requirements. To bypass them, a1qa became familiar with Accertify regulations and how to configure them. Finally, the engineers updated their work environment and Accertify’s settings in such a way that allowed the tests to run without any blockages.
Data glitches
They include desynchronization in the interaction of SAP ECC and SAP Commerce Cloud. The data from SAP ECC is transferred to SAP Commerce Cloud at night. Sometimes, the request for a product chosen from SAP Commerce Cloud can’t be processed as SAP ECC states that the item is out of stock. This results in a failed test.
Content issues
They involve cases of incorrect or insufficient data filling while working on the admin panel. As a result, products lack descriptors, which also causes errors during test runs.
To help minimize the influence of content and data on the stability of automated tests and reduced the number of false positives, a1qa introduced automated scripts (that created the selection of products from SAP ECC, SAP Commerce Cloud, and websites), analyzed them, and presented a list of items that lacked sufficient data for carrying out tests.
