Interview with Daniel Knott: upside down testing

In the second part of the interview, Daniel Knott clarifies the difference between the traditional test pyramid and a mobile one. He also talks on smartwatch app testing, which is the main subject of his recently published book. If you’ve missed the first part of the interview with Daniel on modern trends and challenges in mobile testing, you can find it here.

Daniel Knott is a well-known mobile expert, speaker at various conferences and a blog author. He has been working in the field of software development and testing since 2003.

Daniel, in your book Hands-On Mobile App Testing you write about a testing pyramid. Can you tell our readers about this pyramid?

Sure, in chapter 5 of my book “Mobile Test Automation and Tools” I wrote about test automation pyramids. I think every software tester and developer knows about the testing pyramid created by Mike Cohn. In this pyramid the foundations are unit tests, the middle layer are integrations tests and at the top of the pyramid there are end-to-end tests.

The size of the layer should be an indicator for the amount of automated tests that should be written. There should be lots of unit tests, because these tests are small and fast in execution. Then write many integration tests to test the integration of smaller units. In the end, write some end-to-end tests to see if the system works as expected through every layer.

However, this is just a theoretical model and is a mere guidance for software developers and testers. From my point of view, this pyramid is not applicable for mobile apps and the mobile environment.

Can you please clarify this? What is a mobile test pyramid and how is it different from the traditional testing pyramid?

In most cases mobile apps are used on the move. The users maybe walking around, the commuting to work by car, train or plane.

In all these environments they have different internet connection and face other factors, like weather conditions, that may influence the mobile app functioning. Because of these facts, I think, the traditional testing pyramid is outdated for mobile.

From my point of view, the foundation of the mobile pyramid must be manual testing! Manual testing is the key for a successful mobile app. It is important to test the app in the environment where the potential customer will use it. Simulating the environment is a bad idea and will lead to apps with really bad quality.

The next layer of the mobile pyramid is end-to-end tests covering all components of the app: backend, network and user interface.

The next layer is beta testing – another manual testing layer but with real customers. Try to establish a beta testing community with real customers to get real feedback before going live to 100% of your customers.

And the top of the pyramid there are unit tests. This might sound really strange but writing unit tests for mobile apps is not as easy as for backend or web applications. There are so many APIs and sensors that can be used by an app and it is really difficult and time consuming to mock all those interfaces to write effecient unit tests.

However, I created the mobile test pyramid in a dynamic way like the whole mobile world. The pyramid may not fit every app and therefore it is important to have a flexible model, too.

Daniel, this year you’ve published another book – Smartwatch App Testing. In this book you provide readers with an overview of the different smartwatch platforms with a focus on design guidelines, input methods, connectivity, manufacturer as well as software features. Why have you decided to cover this topic?

It was never my goal to write this eBook in the first place. My initial intention was to learn something new in the mobile sector. I was really curious to learn more about smartwatches. I got my very first one at the end of 2015. It was Samsung Gear S2 and I really liked it and soon I was interested to learn more about watchOS, Pebble OS and Android Wear.

Luckily, I had a chance to use every system for a while and try them out. And as a blog author, I decided to write a series about it on my blog to share my findings with the community. At the end of the series I thought this might be a good idea to publish it on LeanPub as a small eBook.

What are the peculiarities of smartwatch app testing?

Most smartwatch platforms are really stupid in terms of features and functionality without a paired mobile device. So one of the main challenges is the connectivity between the watch and the paired device in order to exchange data.

Another important point is the usability of the smartwatch apps. Smartwatches offer only a very small screen to show the information to the users and because of that, the UI must be well designed in order to perform the required actions efficiently.

There are new ways of interacting with the watch, e.g. with new wrist gestures or voice commands. During my exploring activities I focused on all smartwatch platforms I identified 4 key areas when it comes to smartwatch app testing.

• Design
• Usability/Interaction Design
• Functionality
• Connectivity

What advice can you give to mobile testers?

Try out many things, install many apps on your private device, use them and try to learn from other apps. Stay hungry and up to date to new trends in mobile.

Thanks for for sharing your viewpoint with us. We will be glad to see you and talk to you again.

If you want to learn more from Daniel, visit his blog Adventures in QA. You may also find helpful his books Hands-On Mobile App Testing or Smartwatch App Testing.