Alisa Hrustic
System Verification (Bosnia & Herzegovina)
Breaking the Black Box: How Do We Test AI Systems?
Quality assurance is entering a new and transformative era. Traditional testing approaches, built around deterministic systems with clearly defined inputs and expected outputs, are no longer sufficient. How do we ensure reliability when behavior is probabilistic rather than predictable?
This talk explores how software testing must adapt to address these challenges and introduces a structured approach to AI quality assurance.
Session timeline:
- The session begins with a clear comparison between traditional and AI-based systems from a testing perspective
- AI Testing Pyramid: in AI systems, quality is not only a property of code, but of data, models, and behavior in real world contexts.
- Tools and techniques: It explores how existing QA tools can be adapted
- Hands-on example based on a widely used AI chatbot system. Exploring new types of test cases covering exploratory testing, data validation, and performance testing.
- Automation testing for AI systems: addressing one of the most common challenges teams face today. It explores how traditional automation approaches evolve in the context of AI, shifting from strict equality-based assertions to more flexible evaluation strategies based on score thresholds and behavioral rules.
Comprar Tickets
My name is Alisa Hrustic, and I work as a Test Manager at Alfa Laval.
I hold a Master’s degree from the Faculty of Electrical Engineering in Sarajevo and have over 15 years of experience in test automation and test management.
Throughout my career, I have focused on driving technical improvements and building effective testing strategies that increase team efficiency and product quality. I have worked across a wide range of projects, including API, UI, and mobile testing, gaining hands-on experience with diverse domains and complex system architectures. I am proficient in C# and Java and have extensive experience designing and building test automation frameworks from scratch, from tooling selection and architecture to implementation and long-term maintenance.
In recent years, I have been part of an AI-focused team, where I explore and apply AI-driven solutions to enhance quality assurance processes. My current focus is on leveraging real user behavior to improve test coverage and evolve traditional testing into a more intelligent, data-driven practice. I am an ISTQB Certified Tester – Foundation Level, ISTQB Certified Agile Tester, and ISTQB Certified Tester in AI Testing.
