Leads
Alisha Bose
Tia Jain

Members
Amrit Randev
Jack Le

Mentors
Michelle Hui (AI Studio TA)
Piyush Ghai (Challenge Advisor, Relativity)

Overview

This project investigates the potential harms of generative AI in the justice system, particularly focusing on biases in frontier models. By examining how these biases manifest in the collection and analysis of text, photo, and audio evidence, we aim to ensure equitable outcomes in legal contexts.

Objectives and Goals

Goal: Evaluate the bias of the Gemini API across sensitive topics like disability, nationality, ethnicity, and age using the Amazon Generalized Fairness Metrics dataset.

Impact: Address systemic biases in industrial ML models that affect real-world decisions, mitigating risks that have historically skewed justice outcomes.

Techniques: Explore generic, zero-shot, and few-shot prompting to identify optimal bias detection strategies.

Methodology

Data Preparation

• Downloaded the dataset and removed duplicates or missing values.

• Created structured CSV files for efficient storage and analysis.

• Generated templated sentences for sensitive categories using the dataset.

Modeling & Evaluation

• Developed bias detection and sentiment analysis models.

• Compared zero-shot, few-shot, and generic prompting techniques.

• Used performance metrics like accuracy, F1-score, and precision-recall to evaluate model outcomes.

Tools and Libraries

• Gemini API for sentiment analysis.

• Amazon Generalized Fairness Metrics dataset.

• Python libraries for data preprocessing and visualization.

Results and Key Findings

Key Performance Metrics

Insights

• Few-shot prompting outperformed other techniques, particularly for disability and ethnicity categories.

• Zero-shot prompting exceeded expectations but struggled with nuanced interpretation.

• Models often overemphasized negative terms, impacting performance.

Limitations

• Limited sample size per category due to resource constraints.

• Potential human bias in "true" sentiment labels.

Visualizations

• Performance Comparisons: Confusion matrices and precision-recall graphs for each prompting technique.

• Sentiment Analysis Examples: Highlighted model disagreements and their rationale for true vs. predicted labels.

• Category Analysis: Side-by-side comparison of results across different categories and techniques.

Next Steps

• Extend analysis to more categories and modalities (e.g., images, speech).

• Experiment with advanced prompting methods like Chain of Thought reasoning.

• Upgrade resources (e.g., Gemini Pro) to increase sample size.

• Investigate real-world applications beyond justice (e.g., credit screening, hiring).

Installation

Clone the repository:

git clone https://github.com/example/repository.git](https://github.com/alishabose/Relativity-1D)

License

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software.

Resources

• Amazon Generalized Fairness Metrics dataset

• Gemini API for bias detection.

Final presentation here

Special thanks to Break Thru Tech and Relativity.