Project Equitable Sensemaking: Customizing Fairness and Representation in Crime-Solving Applications Amidst Inherent Data Biases Program DUIRI - Discovery Undergraduate Interdisciplinary Research Internship Term Summer 2024 Status Accepted Research Area Human-AI Interaction Description We're looking into how we can make sure that when we use data to solve crimes, we're being fair to everyone involved, even when the information we start with might already be unfair or biased. We want to figure out how to adjust our approach to fit different situations and find ways to fix these biases so that everyone gets a fair shot and the results are trustworthy. Supervisor Tianyi Li Mentor Tianyi Li Type of work required ataset Compilation: Assist in gathering and organizing mystery-solving datasets that include a wide range of suspects with diverse characteristics. This would involve sourcing datasets, possibly creating synthetic data to ensure diversity, and preparing the data for experimentation.

Attribute Categorization: Work on categorizing the attributes of suspects in the datasets, such as race, gender, criminal history, etc. This task would involve understanding how to classify these attributes in a way that is respectful and useful for analysis.

Experiment Setup: Help in setting up the experiments by defining different permutations of suspect characteristics. This would involve understanding the experimental design, creating clear definitions for each permutation, and ensuring that the setup allows for a fair assessment of the LLM's performance.

Data Annotation: Annotate data or review annotations to ensure that the information fed into the LLM is accurately labeled and reflects the diverse attributes of suspects. This task is crucial for maintaining the integrity of the experiment.

Running Simulations: Assist in running the experiments by feeding the prepared datasets into the LLM under different conditions and recording the outcomes. This could involve using software tools and platforms where the LLM operates.

Results Analysis: Contribute to analyzing the results of the experiments to identify patterns, anomalies, or biases in the LLM's performance across the various permutations of suspect characteristics. This would involve statistical analysis and the ability to interpret data in the context of bias and fairness.

Documentation: Help document the experimental process, the methodologies used, the results obtained, and any insights or conclusions drawn from the analysis. This would involve writing clear and concise reports that summarize the findings and their implications. Websites https://buildingpath.herokuapp.com/step3/user/1/ Qualifications Mystery-Solving Enthusiasm: A keen interest in solving puzzles, mysteries, and complex problems. Sensemaking Curiosity: A strong curiosity about how information is processed and interpreted, both by humans and AI systems. Analytical Mindset: An interest in diving deep into data, discerning patterns, and deriving insights from complex datasets. This involves not just looking at the numbers but understanding the stories they tell and the implications they have for fairness and bias. Ethical Sensitivity: A vested interest in the ethical dimensions of technology and data use, especially as it pertains to fairness, equity, and representation in AI-driven applications. Programming competency: students should be comfortable with developing a customized LLM agent for mitigating biases in sensemaking Credit Hours 2 Weekly Hours 10 (estimated)