Evaluating AI Summaries for Students with Disabilities
This study evaluates how effectively AI-generated summaries simplify academic content for college students with disabilities.
Role
Lead Researcher
Industry
Generative AI
Duration
3 months to present
Project Overview
This project will examine whether AI-generated summaries actually make academic reading easier for college students with learning disabilities. While AI tools are increasingly positioned as study aids, there is limited understanding of how accessible their summaries truly are for students with diverse cognitive needs. Rather than focusing only on technical accuracy, this work will center how students experience and use AI summaries in practice, asking whether they feel clear, usable, and supportive or confusing and overwhelming.
This project is in the process of getting funding and IRB-certified.
Research Goals
The goal of this project will be to understand how different AI summarization tools and prompting strategies shape students’ comprehension and reading experience. The study will compare summaries across clarity, completeness, and ease of use, with particular attention to what makes a summary feel accessible. A central question guiding this work is whether tools or prompts designed with learning needs in mind will produce summaries that students find meaningfully more helpful.
Background and Motivation
Many students with disabilities rely on summaries, simplified text, or alternative formats to keep up with college coursework. Existing research on AI summarization in education often prioritizes technical performance or evaluates user experience in isolation. Few studies compare multiple tools side by side or examine how prompting choices influence accessibility. By combining technical evaluation with direct student feedback, this project will address that gap and contribute to work at the intersection of accessibility, disability studies, and AI in education.
Study Design
This study will use a mixed-methods approach that combines hands-on interaction with AI tools and qualitative interviews. Participants will take part in a structured session that begins with a conversation about their experiences reading academic material and using AI tools. They will then participate in a guided workshop where they review AI-generated summaries of academic passages and compare them based on clarity, completeness, and usefulness. The session will conclude with a reflective discussion about what worked well, what felt confusing, and what features an accessible summarization tool should prioritize. Sessions are expected to last approximately one hour and will take place either in person or over Zoom.
The second part will be a quantitative analysis. Through testing zero-shot vs. few-shot prompts tailored to different learning profiles (e.g., “Summarize this for a student with ADHD”), this project will generate new insights into prompting strategies and their influence in generative AI outputs.
Participants
The study will involve up to 20 University of Washington students who self-identify as having a disability. This sample size aligns with small-scale accessibility research and is intended to support in-depth qualitative insight alongside comparative analysis of AI outputs.
Recruitment and Context
Participants will be recruited through campus mailing lists, disability-focused student organizations, flyers, and word of mouth. Research sessions will be held in quiet, private settings, with flexible pacing and format to accommodate different access needs.
Data and Evaluation
The study will collect interview audio, participant rankings of summaries, written or typed workshop responses, transcripts or researcher notes, and the AI-generated summaries themselves. The research team will also conduct technical evaluations of the summaries, including checks for accuracy, readability, and retention of key concepts. Data will be stored securely and de-identified during analysis.
Potential Outcomes
Through this study, I expect to learn not just which AI-generated summaries are technically accurate, clear, and complete, but also how students with disabilities perceive and actually use these tools in practice. Potential outcomes include identifying which AI tools and prompting strategies best support comprehension, highlighting gaps where AI may oversimplify or omit key concepts, and uncovering usability challenges that aren’t visible through technical evaluation alone. Reflecting on the process will also help me understand the balance between tool performance and human experience, showing how accessibility is shaped by both algorithmic output and individual learning needs. Overall, the project could inform future design of AI tools that genuinely enhance learning for students with disabilities, while also deepening my own understanding of inclusive technology research.