EDU EPSCoR DCL: Towards an assessment framework for engineering students' ethical AI literacy

Artificial intelligence (AI) is becoming an integral part of STEM education, engineering education, and professional practice while also introducing ethical challenges, including algorithmic bias, privacy risks, limited transparency, and unintended societal consequences. Educators continue to lack a unified understanding of ethical literacy needed for engineering students to use AI responsibly. This project will address the critical need of establishing ethical AI literacy as a foundational area of inquiry. A validated assessment framework will be developed as part of this project to enable ethical AI literacy to be systematically investigated. This project has the potential to strengthen engineering education, STEM education more broadly, to support the responsible integration of AI into STEM practice, and to prepare a future workforce capable of developing and using AI in ways that benefit society. The project will establish ethical AI literacy as a theoretically grounded, operationalized, and measurable construct for educational research and inquiry. Grounded in Rest’s Four-Component Model of Morality and leveraging established AI ethics frameworks, this project will operationalize AI literacy into observable competencies as well as develop and validate innovative assessment methods based on student-generated causal maps. Unlike traditional self-report measures, student-generated causal maps externalize how students organize, connect, and apply ethical concepts while reasoning through authentic AI dilemmas. The causal mapping instrument will be iteratively developed and validated through a multi-phase design-based research study. First, a Delphi study with interdisciplinary experts in AI ethics, engineering education, and assessment will establish a consensus-based core set of ethical AI concepts to ensure construct validity and domain relevance. Student-generated causal maps will then be created, piloted, refined, and validated through implementations with undergraduate engineering students. Data will be analyzed using rubric-based scoring, graph-theoretic network metrics, and qualitative analysis, with multiple sources of validity evidence. Overall, the project will generate foundational knowledge about ethical AI literacy, advance assessment methodology, and produce openly available resources to support future research, curriculum development, and faculty practice on ethical AI learning and development across engineering and related STEM disciplines. This project is supported by NSF’s EDU Core Research (ECR) program and NSF's IUSE: EDU Program. The IUSE: EDU Program supports research and development projects to improve the effectiveness of STEM education for all students. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field. Investments are made in critical areas that are essential, broad and enduring. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.