SaTC: CORE: Small: Investigating and Mitigating Harmful Design in User-Generated Virtual World through Design Moderation

User-generated virtual worlds are three-dimensional spaces that offer an immersive experience and are designed and populated by tens of millions of users, including a significant number of children. Many children spend considerable time in these online spaces to create, socialize, learn, and play. Although online virtual worlds represent new ways for children to create content and interact with each other, they also have given rise to harmful design. Harmful design can affect unsuspecting child users in manipulative, insidious ways, such as inducing them to perform violent role-plays, exposing them to abusive content, and tricking them into excessive spending. Previous research has studied how to detect and moderate harmful user-generated content, such as hate speech and harassment, which typically exist in static forms such as text, image, and video. However, limited attention has been paid to harmful design in user-generated virtual worlds. This project aims to tackle this challenge by employing a mixed-methods, human-centered design approach. The objectives are to provide comprehensive empirical and conceptual insights into harmful design, foster broader societal and academic awareness of this challenge, and explore strategies to mitigate this challenge and alike. To achieve these goals, the project is identifying and characterizing harmful design in user-generated virtual worlds and exploring moderation strategies that can mitigate harmful design. The project uses a combination of qualitative and quantitative methods to document and analyze instances of harmful design, as well as how child users experience and perceive harmful design. The project also uses cooperative inquiry to work with children and design experts to develop design guidelines that can promote safety design and mitigate harmful design. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.