This session explores advanced testing strategies essential for maintaining quality in dynamic, rapidly evolving web environments. Increasingly, companies are using sophisticated A/B testing on their platforms, making the testing process significantly more complex and time-consuming. 

How does this impact test automation specifically? A/B tests can affect UI and UX, and may even involve deploying new microservices, multiplying the testing effort. We will use real-world examples to illustrate these challenges and demonstrate how to tackle them with viable solutions at various levels—from easily implemented integration tests with Karate to managing variant activation in end-to-end test execution—with the goal of enhancing test coverage in these constantly changing environments.

Using a case study approach, this presentation details trivago’s journey over several years in addressing the automated testing of multiple web application variants. We learned valuable lessons, including from initiatives that didn’t initially succeed.

We will explore:

• The feasibility of testing everything
• Learning from unsuccessful initiatives
• Developing a holistic approach to the problem
• Implementing “Shift Left” solutions (earlier testing)
• Implementing “Shift Right” solutions (production testing/monitoring)
• Keeping tests up to date with the new ‘default’ experience

Attendees will gain insights into practical strategies for improving automated testing effectiveness in A/B testing-heavy environments.