Research Initiation: Exploring Generative AI for Problem Formulation in the Professional Formation of Systems Engineers

It is not yet understood in what ways and how profoundly the current use of generative AI (gen AI) impacts how students learn to formulate problems. Problem formulation tasks, such as requirements development, define and frame the problems that engineers solve. As such, problem formulation skills are critical to acquire during the professional formation of engineers. In this context, gen AI has been suggested to worsen the difficulty in teaching problem formulation by providing mechanisms for engineering students to cognitively offload tasks, potentially circumventing intended learning. Problem formulation is a complex, higher-order creation task, requiring deep understanding and engagement. The future engineering workforce is currently being prepared in a new gen AI age without evidence for teaching problem formulation with this novel technology. If future engineers are unprepared to formulate problems, significant impacts will be observed in practice. The national imperative to educate proficient engineers with the technical skills necessary to engineer complex systems depends on understanding how gen AI impacts learning to formulate problems. This project will provide foundational evidence on how learning to formulate problems is impacted by gen AI use. The findings of this project will enable new educational approaches and assessments for problem formulation, ensuring future engineers graduate with the necessary technical expertise to engineer future systems that address critical national challenges. This project aligns with the program funding goals for advancing the professional formation of the engineer and expanding the community of engineering education researchers. Simultaneously, the project addresses the critical national funding priority to understand the economic and societal impacts of growing AI adoption. The goal of this project is to understand the impacts of gen AI on how engineering students learn to formulate problems, operationalized through a requirements development task. Four motivating questions will drive the research: 1) Does gen AI usage alter which cognitive problem formulation tasks students engage in when developing a requirements set?; 2) Does gen AI usage alter the quality of the requirements sets developed by students?; 3) Does gen AI usage impact students’ acquisition of core competencies for requirements development?; and 4) Does gen AI usage impact students’ abilities to evaluate their own performance on requirements development? This project will engage in a mixed methods approach to generate qualitative and quantitative data. Three foundations will be novelly leveraged together: (a) integration of systems engineering and social science foundations to enable a conceptual framework of gen AI impacts in problem formulation; (b) comprehensive analyses of the impacts that enable understanding in multiple aspects of learning; and (c) human-subjects research to study common requirements education tasks such as development and evaluation with systems engineering students. Establishing the impacts of gen AI on learning how to formulate problems will enable future research to formalize evidence-based educational practices that balance appropriate human-AI task distributions to the educational benefit of the imminent engineering workforce. By examining the impact of this new technology on the most critical enabler of engineering, the professional formation of the engineer, this project will seek to ensure that future engineers can still successfully formulate problems for real-world systems in the gen AI age. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.