Portfolio
Full-cycle testing of Magento e-commerce solution
The client reached out to a1qa to perform full-cycle testing of the solution and ensure smooth data migration to Magento Enterprise Edition.
Overview
The customer is the owner of a large US-based e-commerce solution specializing in selling natural foods, pet care goods, household cleaners, and other organic products.
As the customer’s business was growing fast, the following needs arose:
- It was vital to perform faultless migration from Magento Community Edition to Magento Enterprise Edition.
- The adaptive website design had to be implemented.
- The already developed functionality needed improvement.
Services offered
Functional testing
Compatibility testing
GUI testing
Mobile app testing
Performance testing
Full-cycle testing
Migration testing
Project scope
Considering the customer’s business needs, the following types of software testing were to be performed:
- Full-cycle testing of the new and already existing functionality after the migration to Magento EE.
- User interface testing after the development of the adaptive design.
- Browser compatibility testing.
- Testing against a number of mobile devices.
- Data migration testing. All the information about the online orders was migrating to a new database. Therefore, it was vital to make sure that all orders had migrated and no data had been lost on the way.
- Load testing to generate user demand on software to measure its response time.
To face the challenge, a1qa was invited.
The customer had no QA processes in place, and the workflow had to be tuned up from scratch. Also, the development team disrupted the deadline, and the communication was hindered by the poorly collected and described functional requirements that had to be specified.
As a result, there was little time left for testing and software stabilization.
To assure the pre-release quality of the business critical functionality and fix the most serious defects, the team was expanded. The specialists were working extra hours and on weekends to complete testing on time.
Initially, the team was made up of 2 engineers and was further scaled up to 4 specialists.
The a1qa engineers had the expertise to fulfill all testing tasks and prepared a detailed roadmap on how to allocate the client’s QA budget for the most effective results.
Functional testing
Both modules placed on frontend and backend parts were tested.
The solution client-side was responsible for user registration, catalog navigation, search and filters, adding products to the cart, promotional codes application, special offers, placing orders, payments, bonuses, and all kinds of notifications.
Backend functions included catalog and products settings, creation of marketing offers and discount terms, stock management, and many more.
The engineers tested both parts and revealed a large number of defects. Over 50% were of the major severity and higher and could have resulted in negative user experience after the release.
The most important issues were connected with the payment option and prices display.
Browser compatibility testing
As soon as the project initiated, the customer made a decision not to support IE 8 and outdated versions of other popular browsers.
The tests were performed in the latest versions of the following browsers:
- Internet Explorer
- Google Chrome
- Mozilla Firefox
- Apple Safari.
Testing against mobile devices
To check the quality of the new design, it was decided to test the solution against four mobile devices (iPhone, iPad, Samsung Galaxy, Samsung Galaxy Tab).
Performance testing
The objectives of the solution performance testing were to:
- Find out the maximum number of concurrent users performing typical operations that the website could handle
- See what would happen when the system became overloaded (i.e. would it fail or just slow down)
- Find out which of the system components (web server, database, or load balancer) was its weak point
- Collect hardware performance statistics of all the servers and see the correlation with the user load
- Measure response times for all the common transactions, find out the slowest ones
- Collect errors statistics.
During testing, all the tasks were fulfilled.
The engineers also concluded that the pages of the website were quite heavy. Optimization of the client-side (leveraging browser caching, images and code structure optimization, JavaScript and CSS shortening, etc.) was recommended.
Test artifacts
Upon the project completion, the customer was provided with the following test artifacts:
- The Quality Report with the description of all defects by their priority, devices, and functional modules of the app where they were detected
- Detailed issues listed in a bug-tracking system, including screenshots and video clips
- Report on the performance testing results
- Structured recommendations on the system environment and app quality improvement.
Technologies & tools
- Magento EE
- Zoho BugTracker
- iOS
- MySQL
- Web
- Android
Results
-
The migration to Magento EE platform was successfully completed, and the product of decent quality was released to the market right on schedule.
-
All high-priority issues were resolved before the release. Under a tight deadline, the testing engineers managed to detect and describe all defects of the major, critical, and blocker severity levels.
-
The client expressed his satisfaction with a1qa work, dedication, personal attitude, and a proactive approach.
In numbers
5
months of the project duration
4
full-time QA engineers involved in the project
4
mobile devices used for testing
50%
of the defects found were of major severity and higher
100%
delivering on time