Encouraging exploratory tests. Interview with Alan Richardson
Alan Richardson has more than twenty years of professional IT experience, working as a programmer and at every level of the testing hierarchy from tester through head of testing. Author of the books “Selenium Simplified” and “Java For Testers”. Alan also has created online training courses to help people learn Technical Web Testing and Selenium WebDriver with Java. He works as an independent consultant, helping companies improve their use of automation, agile, and exploratory technical testing. Alan posts his writing and training videos on SeleniumSimplified.com, EvilTester.com, JavaForTesters.com, and CompendiumDev.co.uk.
– I do agree that it is more a mindset than a methodology. I can’t point at any mandated ‘rules of exploratory testing’ document in the practices of QA outsourcing and say “only if ye do such that is written here shall ye be of the Exploratory”.
Exploratory Testing requires each tester to learn how they currently test, identify how they can improve how they test, and how they can apply their unique skills and experience in their exploratory testing. It is a uniquely personal approach and as such ‘mindset’ offers a far better word to encourage ownership of the testing process than methodology.
Methodology suggests that ’some other’ authority knows best. Exploratory Testing, with its emphasis on learning from the actions and testing that you personally take, can’t fit into that schema.
Those testers I know that excel at Exploratory testing, do so, because they have made it their own. They have looked around at the information that other people have shared about exploratory testing. They have tried it. They have analysed what worked for them and what didn’t. They have adjusted. They have incorporated techniques, approaches, analogies, and skills into their testing that they recognised as valuable to them. And they continue to experiment with, and explore, new approaches and nuances.
Exploratory testing is a learning by doing approach. Therefore we should expect everyone doing it to learn something different and approach it subtly (or sometimes radically) differently.
When I talk to testers I very often talk about taking responsibility for their testing. And by that I mean owning their approach, so that they can justify it using their words, and from their experience, and from the experiments that they have conducted during their testing. I think that is how you build experience, attitude, learning, belief systems; and all of that, I can see falling under the banner of ‘mindset’.
– Selenium WebDriver is a tool that immensely accelerates testers` work. Still, what are the main difficulties that users face while applying the tool?
– I think that Selenium WebDriver offers testers the best open source browser automation library available today. And that when a robust set of domain specific automation libraries have been built by the team, it can allow team members to rapidly automate the functional flows through their application. And that allows teams to build sets of methods which they can use to check the application for unexpected deviations in behaviour. They can also rapidly build up adhoc code which exercises the application until the application achieves a specific state that we can manually test from. The team can also more easily add data into the system, and generally automate flows through a web application.
Common difficulties that people have when they start using Selenium WebDriver is viewing it as a tool. I view Selenium WebDriver as a library, and like any other library that we use when programming, we have to know how to program to be able to use the functions in the library. So programming knowledge is required. And that can prove a difficulty for people.
The Selenium eco-system does provide tools; like the Selenium IDE, or Selenium Builder. Which allow recording of actions with the automatic generation of scripts. But unless you know programming, you don’t know how to effectively tailor those scripts, and wean yourself off the recording tool.
Other difficulties people have, relate to the web technology itself, simply because there is a lot to learn if we really want to master automation.
Because we use Selenium WebDriver to automate the browser, we have to understand some of the technology associated with how browsers work; the DOM, CSS, JavaScript, etc. We have to understand how the specific pages we are testing work, so that we know how to synchronise effectively with the state of the page so that our automation works consistently.
So I find that, over time, to effectively work with Selenium WebDriver, we have to learn: programming, CSS selectors, HTML, DOM, dynamic HTML population, synchronisation strategies, browser developer tools, software design, continuous integration, environment maintenance, etc. (the list could go on).
I don’t think these learning topics are unique to Selenium WebDriver. I think they are important for any browser automation. But I think some automation technology and tools use a sales tactic of pretending that their users will not need this learning, but they will.
You can achieve good and effective results with minimum levels of knowledge in these areas, but to really make your automation work, you may have to learn things that other testers, may not immediately need to learn.
I could continue to list many more things that people find difficult because I do receive a lot of emails from people learning Selenium WebDriver and the issues they are having.
Very often someone on the web has already faced, and identified a workaround to, their issue. So sometimes the difficulty people have is that their google-fu isn’t quite effective enough to find the existing answers out there on the web that will help them.
– You do a lot of webinars. Is it your pass
– I need to do more webinars. I don’t do enough of them.
I like to share experience. Partly because I learn when I do that, because I have to codify it in a different way to communicate it to others. Partly because you can reach more people over time, with a webinar, than through a conference talk.
A conference talk will reach several hundred people, at a single event, and that is it over, done. I might release the slides and a blog post, but it doesn’t have the power of the presentation behind it.
With a webinar, and online training, I can reach thousands of people. And they can encounter it at the point when they are ready for it.
We also give the viewer more control: they can pause it, rewind it, or skip sections if I’m wittering on about something unimportant to them. Webinars and videos put more control in the hands of the viewer, than in the hands of the presenter, and that allows the viewer more control over their learning process.
Also the commenting systems online give people the opportunity to think about questions, ask them whenever they want to, and have more of a discussion than the conference format often allows.
Webinars also allow me to ‘demonstrate’ stuff more than I can in a conference talk, and I think it is very important for people to see things in action. Because it shows what is possible, and that can help change a belief or a mindset about what ‘testing’ means, or about what is acceptable in testing. And they can model the behaviour shown and achieve the same result.
And I guess I’m passionate about that.
– Will it be beneficial for the workflow, if team combines exploratory testing with agile techniques?
– Exploratory testing allows us to test at speed, often with minimal information, it encourages learning through investigation and conversation, we can collaboratively make decisions about which risks to explore in detail and which to perform less work on. Every Agile process I’ve worked in, has required the associated testing process to have those attributes.
I have worked on projects where people have expressed the opinion that the Acceptance Criteria on a story is good enough to assess the completed code against. So the team then build confirmatory Acceptance Automation – possibly using BDD, or ATDD, or by building Gherkin feature files or xUnit tests to cover the criteria. And when they all run green the team consider the Story done.
This is a confirmatory process.
We also need a questioning process. “Did you mean X or Y?”, “Did you only mean X?”, “What if Z happens?”, “What about…?” etc. etc.
Some of those questions may occur during the discussions as the story is coded, and some of the answers may even be added as Acceptance Criteria.
But I’ve seen too many stories where questions were not asked.
Even when the questions were asked, the answers weren’t written as acceptance criteria, and so the confirmatory automation did not cover them, and sometimes the functionality didn’t cover the criteria effectively, but no-one knew until the software was released.
An exploratory testing process can ask those and other questions.
But the main reason I think we have to combine exploratory testing with agile techniques is because all testing is exploratory. So if you want to actually have testing on your Agile project, you have to explore.
– Alan thank you for sharing your ideas and expereince. we hope to talk to you again.
