Equity and AI | UDaily

When ChatGPT burst onto the scene in November 2022, many teachers and parents worried that the new artificial intelligence (AI) writing tools would help their students miss important learning opportunities. Instead, as University of Delaware associate professor Joshua Wilson has shown, AI writing tools and assessments actually helped students develop their writing skills and supported teachers in providing meaningful feedback.

Now, in a recent study published in Science and teachingWilson and his coauthors focus on elementary English learners (ELs), examining how this growing population of learners interacts with and uses automated writing assessment (AWE) software. They find that AWE technologies are equally beneficial for ELs and non-ELs. Participants in the study received similar levels of feedback on their writing, made similar gains in writing quality, focused on consistent dimensions of writing during revision, and endorsed AWE to similar degrees, regardless of their language status.

“As AI-based feedback applications become more common, it is critical that researchers examine the implications of implementing these tools in authentic learning environments with a focus on equity,” said Wilson, who specializes in literacy in the College of Education and Human Development (CEHD) at the University of Delaware. “This study represents a novel step forward in the field of AWE because it focuses on multifaceted student engagement and ensures that there are no systematic differences in engagement that could disadvantage vulnerable subgroups. This approach sets a precedent for other studies of AI-based feedback applications, ensuring that these technologies support equitable learning outcomes for all students.”

Automated Essay Grading Software

AWE refers to a class of educational technology tools that use natural language processing and artificial intelligence to provide students with automated feedback that supports improved writing quality. Wilson’s study focuses on MI Write, an AWE system designed to improve the teaching and learning of writing by providing students with automated feedback and writing scores.

When a student writes an essay and submits it within MI Write, their essay is immediately analyzed by scoring and feedback algorithms, which then delivers immediate automated feedback directly to the student. In addition to providing the student with an overall writing assessment, it also provides specific writing assessment and feedback on idea development, organization, style, sentence flow, word choice, and convention. MI Write also includes peer review opportunities, offers multimedia lessons for skill building, and allows teachers to communicate with students through commenting features.

“(Students) seem more determined and (MI Write) is so tailored to them,” said a fourth-grade teacher in Wilson’s study. “It’s almost like there’s a person, someone (teaching) them, conferring with them, telling them how they can improve, all at once. Whereas before, I wouldn’t have been able to physically do that that quickly.”

Benefits for English language learners

To assess how ELs interacted with and benefited from AWE technology, Wilson and his co-authors collected data from nearly 3,500 students in grades 3–5 in a Mid-Atlantic district that implemented MI Write in all 14 of its elementary schools during the 2017–18 school year. They collected data from ELs—students whose native language is not English and who qualify for English language services—and non-ELs.

To examine interactions with the AWE software, Wilson and coauthors looked at three dimensions of engagement: behavioral, or the actions students take in response to feedback; cognitive, or the thinking and revision strategies students use in response to feedback; and affective, or how students feel and perceive the feedback.

Across all three dimensions, Wilson and his coauthors found similar levels of engagement across all students. For example, some students did not use the feedback provided by AWE, whereas others used it repeatedly. However, these differences in behavioral engagement were not related to language status. Similarly, ELs revised their texts productively to the same extent as non-ELs and often focused on the same set of textual features during revision.