Blog

Keyword-driven testing: reasons to use or skip. Part 1

Automation is a topic that is under constant discussion and improvement today.  Its consistency has been already proved.  But for IT industry in general this practice pretends to be more and more effective and applicable, to fit QA process needs better.  One of its means is cost effective usage of testing. Just imagine that you can double return on investment KPI.
9 September 2014
Test automation
The article by Dmitry Tishchenko
Head of global business development at a1qa

Automation is a topic that is under constant discussion and improvement today.  Its consistency has been already proved.  But for IT industry in general this practice pretends to be more and more effective and applicable, to fit QA process needs better.  One of its means is cost effective usage of testing. Just imagine that you can double return on investment KPI. On the one hand you save money, on the other can increase quality using the same budget or mitigate QA budget risks on stabilization and delivery stage.

There are lots of interpretations of keyword driven approach in the world of test automation. Different tools allow using keywords.  But not all that cases may be a true keyword driven. The way I used to describe it in simple words is: “manual testers mostly create test cases using keywords and test developers create code which implements particular keywords via available interfaces”.

What is coming out that meaning?

  1. The keyword driven approach helps to achieve better ROI in long term saving money on support activities.
  2. It optimizes (moves down) technical threshold for stuff who may be involved in automated tests development process i.e. persons who prepare description of tests which is pretty the same like in test automation in terms of keyword driven. We may say that a person who prepares description of test using already implemented keywords creates automated test the same time.
  3. The last but not least – it maintains the relations between test developers and testers themselves regarding automated tests which positively impacts QA budgets and overall process.

Of course, there are patterns for using that approach like for data driven or other ones we are aware of.  For instance, you may apply scripting technique which requires less money for initial test development, if your test process assumes coverage of stable critical functionality (meaning smoke test) the frequency of builds is closer to daily basis, and testing is required for more than 4-6 upcoming months.

Thinking of Keyword driven approach you should ensure that you have a lot of business logic. There are similar actions which can be used in different test cases (potential keywords), the business logic is too complex for Data-driven approach, which may be extremely effective in some other particular cases.

So, you never know without trying! We decided – it’s worth doing. I’d like to share some of the most important findings. One of the crucial requirements here is that special description format should be applied from the very beginning of testing activities. Taking into account this fact and absence of experience with tools that are specially designed for Keyword driven approach implementation we decided to split our first Keyword driven practice in two steps.

  • Step 1 – Proof of Concept (POC). This is the base for the article you are reading. Mentioned step purposed to try out a particular tool, check basic automation indicators like execution/support efforts, check if manual tester might be able to handle test case description.
  • Step 2 – Using Keyword driven approach. The second one is to apply this approach from the very beginning of the particular project and cover the whole QA cycle using it.

Thus, we have found a “guinea pig” for the first step and here is a description (without mentioning concrete customer, of course).

Solutions comparison

 Existent solutionKeyword driven POC
Automation approachScriptingKeyword driven
Programming LanguageC#Python
Test Automation frameworka1qa SMART frameworkRobot framework
Automation toolSeleniumSelenium
Test execution systemTFSJenkins

Basically, we have selected quite stable scope in terms of business logic and user interface changes. It was decided to select rather large scope which guaranties that there will be enough keywords to make measurements and to check test development abilities using keywords only. As a matter of fact, 200 test cases were selected for automation.

It should be mentioned that selected project was quite dynamic and required daily test executions. It was necessary to speed up solution comparison process.

One of the required factors was to get a situation when test development efforts on the keyword driven test framework are less labor-intensive in comparison with actual automation way for existent solution. We achieved mentioned result just after 150+ cases were developed.

Some good lessons have been also learned about Keyword driven while tests were developing. There are top-3 issues you should consider applying keyword driven practices to the project:

  • TBD 1– First tests on Robot Framework KDT requires quite more than average efforts (5-6 hours when scripts with DDT regular approach requires 3-4 hours). Efforts decrease with list of base keywords being completed and available for re-use (Down to 1-2 hours).
  • TBD 2 – The tester is required to learn several special formats & keywords for creating the customized keywords. It takes some time to get familiar with framework specifics on tests development (IDE, libraries, etc.)
  • TBD 3 – Robot Framework execution engine doesn’t help contain default mechanisms implementing itself to solve parallel test execution problem.  It was quite a big challenge to implement this mechanism. We tried to run scripts simultaneously on a single VM by using several jobs in Jenkins, but it failed because of AUT UI specific interactions: some UI elements were available only in the currently available browser instance.

A soon as a number of tests were ready, we started actual comparison.

In the next post we`ll provide you  with the results we got.

The article by Dmitry Tishchenko was published in UK magazine “European Software Tester”.

More Posts

Enhancing Agile and DevOps processes
28 February 2024,
by a1qa
4 min read
4 actionable tips to enhance Agile and DevOps processes 
Streamlining Agile and DevOps workflows? Learn practical recommendations on how to achieve this.
Agile
General
Test automation
Navigating the future: QA trends that will define 2024. Part 1
15 January 2024,
by a1qa
4 min read
Navigating the future: QA trends that will define 2024. Part 1
Discover topical software testing trends that will shape 2024 and empower companies to smoothly implement advanced technologies.
Agile
QA trends
Quality assurance
Test automation
The year in valuable conversations: recapping 2023 a1qa’s roundtables for IT executives 
8 December 2023,
by a1qa
3 min read
The year in valuable conversations: recapping 2023 a1qa’s roundtables for IT executives 
From dissecting novel industry trends to navigating effective ways of enhancing software quality — let’s recall all a1qa’s roundtables. Join us!
Big data testing
Cybersecurity testing
Functional testing
General
Interviews
Performance testing
QA trends
Quality assurance
Test automation
Usability testing
Web app testing
na-st-awards-23
16 November 2023,
by a1qa
3 min read
a1qa shines as the finalist in three categories of the North American Software Testing Awards
a1qa is a triple finalist at the North American Software Testing Awards.
General
Quality assurance
Test automation
3 November 2023,
by a1qa
4 min read
From idea to buying: 7 testing types to make your mobile eCommerce solutions flawless
Read the article to discover 7 QA activities helping boost mobile eCommerce solutions quality and provide end users with exceptional buying experiences.
Functional testing
General
Quality assurance
Test automation
Usability testing
On the way to Web 3.0: key software testing aspects for seamless digital experiences. Part 2
12 October 2023,
by a1qa
4 min read
On the way to Web 3.0: key software testing aspects for seamless digital experiences. Part 2
Let’s analyze essential software testing checks to improve the quality of the business-critical Web 3.0 functionality.
Cybersecurity testing
Functional testing
Performance testing
Quality assurance
Test automation
Usability testing
On the way to Web 3.0: key software testing aspects for seamless digital experiences. Part 1
11 October 2023,
by a1qa
4 min read
On the way to Web 3.0: key software testing aspects for seamless digital experiences. Part 1 
In part 1 of this article, learn about the transformation to a new Internet era, Web 3.0, and its benefits for increasing operational efficiency.
General
QA trends
Quality assurance
Software lifecycle QA
Test automation
goodfirms-lists
26 September 2023,
by a1qa
2 min read
a1qa’s recognition by GoodFirms: A testament to our expertise
Delve into a1qa's recent acknowledgment by GoodFirms and explore the significance of this accolade in the sphere of software testing.
General
Performance testing
Test automation
ai-ml-test-automation-mini
21 September 2023,
by a1qa
5 min read
6 activities to supercharge test automation with AI and ML
How to maximize the benefits of test automation? Learn how to do it via applying AI and ML.
Test automation

Get in touch

Please fill in the required field.
Email address seems invalid.
Please fill in the required field.
We use cookies on our website to improve its functionality and to enhance your user experience. We also use cookies for analytics. If you continue to browse this website, we will assume you agree that we can place cookies on your device. For more details, please read our Privacy and Cookies Policy.