Wasn't Built for Us: Perception of AI-Supported Writing Technology
Explores the intricacies of text-based digital interactions facilitated by AI-supported writing technologies (AISWT) that have become inherent in our daily lives.
Role
UX Researcher
Industry
Artificial Intelligence
Duration
8 months
Goal
Our goal was to explore how Black users engage with digital technology, focusing on aspects that reflect their real-life experiences and highlighting areas where current digital tech design may fall short. The study delved into Black users' perspectives through interviews, imagined scenarios (design fiction), and firsthand observations, particularly in relation to AI-driven text technologies used in conversations.
Full paper published for ACM SIGCHI CSCW here.
Stage 1: Interviews
In-depth discussions were held through interviews to gauge the integration of AI-supported text technologies with the lived experiences of Black users. The aim was to grasp the participants' perspectives on the Black lived experience, its portrayal (or lack thereof) in their tech interactions, and how AI text tools either aligned with or overlooked their lived realities.
In a subsequent phase, participants engaged in a brief writing exercise using Google Docs to express themselves naturally, followed by reflections on the impact of grammar and spelling suggestions. This was followed by utilizing ChatGPT prompts to continue writing, allowing for examination of how well ChatGPT captured their communication style.
Speculative Design Fiction Workshop
Employing a workshop format, participants engaged in speculative design exercises aimed at reenvisioning the functionality of these technologies. The objective was to foster discussions on their existing deficiencies and explore pathways to enhance their effectiveness for Black users. Through creative brainstorming, participants sought to conceptualize innovative solutions that address the specific needs and experiences of Black users in digital spaces.
Thematic Coding
To understand how participants perceive and experience ASWT, we conducted a thematic analysis of the interview data gathered by our team. We began by cleaning the Zoom audio transcriptions using Otter.AI. Then, we performed inductive coding on two interviews, ensuring a blind coding process for each. This generated an initial list of codes. Next, we collected all the generated codes and merged similar ones. We utilized an affinity map to create broader coding groups.
Findings
Collaborated closely with the development team to ensure a smooth transition from design to implementation. Provided ongoing support and guidance during the development phase, addressing any design-related challenges that arose. Played a key role in the app's successful relaunch, monitoring user feedback and engagement post-launch to inform future updates.
Below is a comparison of Original Story Draft (Left) and AI-Generated Continuation (Right) during Remote Moderated User Observations. Participants engaged in an AISWT task where they first wrote a story in their natural vernacular, prompted by a casual writing prompt. The left side shows the participant's original writing in their natural tone, while the right side illustrates ChatGPT's attempt to continue the story with consistent tone and vernacular, as per the participant's style.
Reflection and Impact
This project underscored how AI text technologies often fail to reflect the nuance, voice, and cultural context of Black users’ lived experiences. Moving forward, this work points toward more participatory, culturally responsive design approaches, where Black users are not just subjects of evaluation but active shapers of how AI systems communicate, adapt, and support expression in digital spaces.