The IndicTTI Benchmark

Navigating Text-to-Image Generative Bias across Indic Languages

This research explores the bias in text-to-image (TTI) models for the Indic languages widely spoken throughout India. It examines and compares the generative performance and cultural aspects of leading TTI models in these languages, contrasting it with their English language capabilities. Employing the proposed IndicTTI benchmark, this research comprehensively evaluates the performance of 30 Indic languages using two open-source diffusion models and two commercial generation APIs. The primary objective of this benchmark is to measure how well these models support Indic languages and identify areas in need of improvement. Considering the linguistic diversity of 30 languages spoken by over 1.4 billion people, this benchmark aims to provide a detailed and insightful analysis of TTI models' effectiveness in the context of Indic linguistic landscapes.

License Agreement + Citation

The code and data for this research can be accessed here.

This database is available only for research and educational purpose and not for any commercial use. If you use the database in any publications or reports, you must refer to the following paper:

Mittal, Surbhi, Arnav Sudan, Richa Singh, Mayank Vatsa, Tamar Glaser, Tal Hassner. "Navigating Text-To-Image Generative Bias Across Indic Languages." European Conference on Computer Vision (ECCV) 2024. (Accepted)

@article{mittal2024indicTTI,
title={Navigating Text-To-Image Generative Bias Across Indic Languages},
author={Mittal, Surbhi, and Sudan, Arnav and Singh, Richa and Vatsa, Mayank and Glaser, Tamar and Hassner, Tal},
journal={European Conference on Computer Vision },
year={2024},
publisher={Springer} }