A Geometric Perspective on ML Safety Assurance

Emmanuel Ledinot (1) , Gassino Jean (2) , Ricque Bertrand (3) , Mekki-Mokhtar Amina (4) , Serratrice Franck (5) , Philippe Quere (6)

1 Thales Research and Technology [Palaiseau]
2 IRSN/PSN-EXP/SSyR/BASEC - Bureau d'Analyse des Systèmes Electriques et du Contrôle-Commande
3 SAFRAN Group
4 ANSYS
5 RENAULT
6 Stellantis France

Some people claim AI-ML suffers from a reliability glass ceiling effect, around 10e-2 per inference, that makes it incompatible with safety-criticality by several orders of magnitude. Others advocate that safety nets and development assurance will overcome this gap so that there is no real concern indeed. We propose an explanation to the reliability plateauing phenomenon based on geometry of approximant adjustment, and on ML verification practices. We advocate the need for a new field we coined as HR ML (Highly Reliable) and UHR ML (Ultra Highly Reliable). Relying on Topological Data Analysis in high dimensions, its aim is to supplement data-science pointbased verification with volume-based verification in order to meet the needed 10e-5 / inf. error rates (and beyond). We argue that process-based ML assurance and safety monitors alone will not overcome the reliability barrier. Our HR-ML concept for safety-related applications is a research proposition at the confluence of ML assurance and system assurance.

Mots clés: machine learning ML reliability safety assurance ML assurance latent manifold extensional coverage analysis Topological Data Analysis (TDA); machine learning; ML reliability; safety assurance; ML assurance; latent manifold; extensional coverage analysis; Topological Data Analysis (TDA)

Domaines: Informatique [cs]; Statistiques [stat]

Emmanuel Ledinot, Gassino Jean, Ricque Bertrand, Mekki-Mokhtar Amina, Serratrice Franck, et al.. A Geometric Perspective on ML Safety Assurance. 2023. ⟨hal-04082469⟩