Hugh McCamphill
Glofox (UK)

Biography

Hugh is an experienced test professional, working in differing testing and test automation leadership roles. He is currently working as a Test Lead with Glofox. He has been the organizer of the Belfast Testing and Automation meetup since 2013, and has previously been a co-organizer of the Belfast Testing Clinic and Test Bash Belfast.

About the full day Tutorial

Getting your automated tests in the pipeline

 

According to the DevOps Research and Assessment (DORA), high-performing organisations release multiple times per day with a lower change failure rate; test automation can be a key component in achieving these twin aims. How do we go about introducing end-to-end automated tests to a CI/CD pipeline that will provide additional levels of test coverage while also not impacting another key metric, lead time for changes?

Two years ago, we were already releasing multiple times per day; however, we can have problems when trying to add these types of tests to a pipeline – including the potential for them to be hard to understand, slow to run, and untrusted by developers (due to flakiness). In fact, many teams end up only running them overnight for these reasons, but this reduces their value (and gives other problems)

Having experienced all of the problems mentioned in the past, I was very clear on the steps required to solve them, and the steps required to have end to end tests included in the pipeline, – required to pass before release, and providing valuable feedback that would not impact on how often developers can release.

* Why you don’t want to run tests overnight

Firstly I’d cover why it’s an antipattern to run end-to-end tests overnight in a continuous delivery environment. Apart from simply being too long to wait when developers can (and will!) release every day, it’s also introducing a clear separation between developers adding features and the writing of end-to-end tests. In addition, when tests fail overnight, it can be hard to trace the release back to the change that caused the failure.

* Gaining trust

Firstly, we built a small number of tests that would be included, without requiring them to pass before release. I then monitored these tests for a couple of weeks until I felt personally felt trust in these tests. Once I did, I could then make the passing of them a required step in the pipeline and feel confident about any release.

* Maintaining trust

After an initial set of tests were in the pipeline, it was important to maintain the trust initially gained. We built a basic automated process that would validate that the tests that were created and ran locally, would also run correctly 100% of the time in a Continuous Integration environment (as far as we could guarantee)

* Making tests easier to understand

We decided on creating abstractions and a Domain Specific Language that would make the understanding and maintenance of tests easier – ie, creating tests in the language of the application rather than the language of the technology.

* Making tests faster to run

**Firstly – Tooling that would scale with your tests

It’s important from the outset to have libraries and frameworks that will handle running tests in parallel. I will share how we built a solution based around WebdriverIO and Jest that allows for tests to be run in parallel. In addition, we used CI tooling that would allow tests to be grouped and run in parallel.

**Secondly – leveraging APIs

It’s important to not just test our APIs, but to use them to set up state and perform validation as part of browser tests, making the browser interactions tightly focused only on the parts we are interested in actually testing through the browser.

Join me as I discuss the steps and approaches above taken to get the tests included so that you too can have quick, reliable test automation in your pipeline