Can We Trust AI Agents to Program and Test Safety-Critical Systems? Current Standards, Qualification Gaps and Emerging Solutions

AI agents are rapidly entering software development and testing workflows. They can generate code, create test cases, perform reviews and automate engineering activities that were traditionally carried out by humans. While these capabilities offer significant productivity benefits, their use in safety-critical development raises an essential question: can such tools be trusted?
This presentation explains how existing functional safety standards, including IEC 61508, ISO 26262 and DO-330, address development and verification tools through concepts such as tool classification, tool confidence and tool qualification. We show how these mechanisms can already be applied to AI-based development tools and where important limitations remain.
Using practical examples from AI coding and testing agents, we demonstrate how safe deployment is possible today through controlled usage, independent verification and formal methods that detect or mitigate AI-generated errors. These approaches allow organisations to benefit from AI while remaining compliant with current safety standards.
The presentation then provides an overview of emerging guidance for AI-based tool qualification, including developments within ISO/IEC TS 22440 and concepts proposed by EASA. We discuss which qualification principles are likely to remain stable, where qualification of AI-based tools differs from qualification of classical deterministic tools, and what practitioners can do today to prepare for future compliance requirements.

Attendees will leave with a practical understanding of what current standards permit, where the qualification gaps are, and how AI agents can be introduced into safety-critical development processes in a controlled and defensible manner.