Students, educators see a path to better writing though KU-developed AI feedback tool

LAWRENCE—While writing skills are critical to academic and post-school outcomes, teaching these skills is a time-intensive process for educators who must read, score, and offer individualized feedback to students for revisions.

That prompted KU Life Span Institute researchers Sean J. Smith, professor of Special Education, and Samantha Goldman, assistant research professor at the Life Span Institute, to pursue a project that could help students who were struggling to meet writing benchmarks. Project AI SCORE, which stands for Artificial Intelligence Scored Composition to Improve Outcomes for Written Expression, uses artificial intelligence to provide feedback on ideas, organization, and style.

Smith said writing isn’t only important for language arts classes. Most if not all subjects require writing — from explaining one’s thinking in algebra to writing essays for history or science.

“We realized it was a problem years ago,” Smith said. “We have teachers that are asking students to demonstrate their knowledge in writing. And it’s not just in English language arts. It’s science teachers with science reports or social studies teachers.”

Now in their second year of a $1.875 million grant from the U.S. Department of Education, the project's research team has been working with teachers across Kansas and the U.S. to test and refine the tool. One of those testing is Lauren Hawkins, Dean of Exceptional Learning at St. Paul VI Catholic High School in Chantilly, Va.

“It provides a nice structure for the writing process itself, from beginning to end, like going through a checklist and making suggestions,” Hawkins said. “We've told the students that it's their writing, but this is just a tool to help facilitate the writing process.”

Educators can access AI SCORE anywhere, whether they're in a rural school district in western Kansas or in large cities on the west coast, said Smith.

AI SCORE builds on the success of the existing, web-based progress monitoring tool developed by KU researchers including Smith, called WRITE PM. School districts have used WRITE PM to help students with learning disabilities improve their writing skills with automatic scoring for quantitative elements of writing such as number of words, word sequence, spelling, and number of letters.

AI SCORE introduces AI tools to provide qualitative writing feedback, focusing on ideas, organization and style, for any student. It begins with an informative writing prompt selected by the teacher. 

“The students have a minute to plan their responses and nine minutes to write,” Goldman said. “Then the program automatically scores the writing and reports areas of strength and targeted, actionable feedback to improve the writing.”

While the project originated in Kansas, researchers have been expanding to schools across the nation at both middle and high school levels, to help students who have learning disabilities and those who don’t.

Hawkins, the Virginia education administrator, described a student with attention deficit disorder (ADHD) who had struggled with turning in assignments, particularly in writing. AI SCORE not only helped the student improve her work, but the instant visual feedback encouraged her to meet her writing goals.

"So, it hit different modalities of learning for her and provided a structure to help her practice writing,” Hawkins said.

Smith said immediate student feedback is key to improving writing, which is why researchers built that into the project.

“They get the feedback immediately, to address misconceptions right away,” Smith said, noting that the need for instant feedback is well demonstrated by other prior education research.

Educators get the results from individual assignments. They monitor student progress and work with students to improve their writing. 

“Educators remain critical to the system. It can’t work without them,” Goldman said. 

AI SCORE prioritizes keeping the teacher “in the loop.” Teachers review and rate the feedback from the AI, which is used to help refine the system. 

“It’s the teachers that meet with students about their data," she added. "It’s the teachers that provide those targeted interventions and strategies that really make the difference."

Hawkins said she's seen how AI SCORE helps teachers make more efficient use of their time, giving them more flexibility to work with students to improve their skills.

“Overall, my impression is this is a smooth process, and it takes the burden off of the teacher to sit there and help facilitate that whole writing process," Hawkins said. 

Initially, as large language models like ChatGPT were released to the public a few years ago, educators were concerned students were using AI to do the work for them.

“The mindset in our building has changed significantly,” Hawkins said. “This is a more meaningful use of AI to give that real-time, live feedback, and it doesn't allow for opportunities to cheat.” 

As the project advances, the researchers are ramping up use in schools across Kansas. 

"We are very Kansas based, and we're trying to promote and support Kansas teachers and schools as much as possible,” Smith said. 

Classroom educators, leaders or district personnel who are interested in collaborating or learning more can ask for more information from the research team.