OpenAI Launches IndQA, a Culturally-Grounded Benchmark for Evaluating AI Across Indian Languages

OpenAI has unveiled IndQA, a new evaluation benchmark designed to measure how effectively AI systems understand questions rooted in India's linguistic and cultural landscape. Unlike traditional benchmarks such as MMMLU and MGSM, which rely on translation or multiple-choice formats, IndQA comprises 2,278 natively-written questions developed by 261 domain experts across 10 cultural areas including literature, history, food, and spirituality.

The benchmark addresses a significant gap in AI evaluation. With approximately 80 percent of the global population speaking non-English languages as their primary tongue, most existing benchmarks fail to capture the cultural context and linguistic nuances that matter for real-world AI applications. India, with over a billion non-English speakers and 22 official languages, represents ChatGPT's second-largest market, making it an ideal starting point for this initiative.

IndQA employs a rubric-based grading system where domain experts establish weighted criteria for each question, ensuring responses reflect culturally-informed expectations. OpenAI's leadership emphasizes the company's commitment to developing language capabilities that serve diverse global users authentically.