When two companies merge, migrating tens of millions of customers onto a single platform is a monumental challenge. In our migration, a decision to move fast—skipping the external beta—left us without the usual qualitative insights from early testing. Our support call queues were unprepared for a new tech stack, and a small canary cohort meant issues went unflagged. Suddenly, the only real signals came from in‐product feedback and uninstall surveys.

To overcome this, we adopted a shift‐right approach, building a lightweight, AI‐driven solution using LLMs. We automatically ingest and categorize real‐time user feedback, then deliver interactive dashboards for testers—scaling to the demands of a live, high‐volume environment. Best of all, this process is simple and replicable in any cloud, with a practical checklist you can immediately apply to your own projects.

In our talk, we’ll share the real‐world case study of how we developed the “User Feedback Overview” LLM‐powered feedback evaluation and monitoring system, ultimately reducing blind spots and improving troubleshooting speed during a massive customer migration. Join us to see how real‐time feedback can transform your testing strategy and delight your customers—even under extreme pressure.

Michal is an Engineering Manager and Product Owner at Gen Digital, where he focuses on developing innovative products that support ARM architecture and NPU-based AI features. With extensive experience ensuring that company products reach over 450 million customers in top shape, he has played a key role in multiple successful launches. His background includes working on diverse projects at companies like Avast Software, Sun Microsystems and CA Technologies. Michal specializes in critical product launch execution, optimizing delivery speed, and reducing release costs. 

Michal is a Software Engineer in Gen Digital’s Windows UI QA team, where he improves the quality of user activity reporting and delivers actionable insights that help uncover root causes of complex UI issues. He also builds a lightweight, custom AI-driven tool to process real-time user feedback and uninstall surveys, enabling cross-functional teams to detect and address emerging issues faster.