I-Corps: Translation potential of Automated Artificial Intelligence (AI) Reliability Evaluation Platform

This I-Corps project is based on the development of an automated platform that evaluates the reliability of artificial intelligence (AI) systems before they are deployed. Currently, there is no efficient, standardized way to confirm that an AI model is reliable, fair, and safe. Errors or unfairness can affect large numbers of people who have no visibility into how the systems were validated. Checking is done manually with fragmented, disconnected tools with no common measure of reliability. This technology automates evaluation, combining multiple checks into a single tool that produces an easy-to-understand reliability score, a risk breakdown, and clear recommendations. The application includes organizations that must demonstrate reliable and compliant AI as regulation expands. By making reliability evaluation faster, standardized, and accessible to non-specialists, the technology may reduce validation cost while supporting the responsible adoption of AI. This I-Corps project utilizes experiential learning coupled with first-hand investigation of the industry ecosystem to assess the translation potential of an automated, domain-aware platform for evaluating the reliability of artificial intelligence (AI) systems. Current approaches are limited to manual self-assessment frameworks, high-level governance guidelines, or single-property toolkits that address only one concern. This technology combines a multi-dimensional assessment pipeline with a weighted scoring framework that evaluates a model across robustness, privacy, fairness, transparency, and accountability, then normalizes these heterogeneous signals into a single interpretable reliability score that penalizes critical failures and adjusts weights by application domain. This unifies translating legal, ethical, and governance requirements into measurable technical tests and combines otherwise incompatible metrics into one repeatable, auditable measure. Users may benefit by replacing slow, subjective, expert-dependent validation with a standardized and repeatable process that produces interpretable, audit-ready results and can be re-run continuously as systems are updated. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.