NVIDIA and IIT Madras Open 2.8T-Token Corpus for 22 Indian Languages

01.A data gap India has lived with for years

Indian-language AI has trailed English largely for one reason: too little training data. Mitesh Khapra, who heads AI4Bharat at IIT Madras, has put the problem plainly, saying Indian language technology trails English because we do not have enough data for Indian languages. On October 16, NVIDIA and IIT Madras' AI4Bharat lab released Nemotron-pretraining-Indic-v1, a corpus of about 2.8 trillion tokens covering all 22 languages listed in India's Constitution, along with the open pipeline used to build it. The release builds on AI4Bharat's earlier Sangraha corpus, which NVIDIA describes as having helped lay the foundation for open Indic language-model research.

The 22 languages are the first languages for 97 percent of Indians, or nearly 1.2 billion people at the 2011 Census, according to the announcement. For a country where most people do not work in English, that is the point: more training material in people's own languages is the raw ingredient for assistants, translators and classroom tools that actually work, rather than tools that only half-understand the person using them.

02.What is actually inside the corpus

The corpus contains 2,601.0 billion native-script tokens and 224.6 billion romanised tokens, counted with the Gemma-4 tokeniser. Native script means a language's own writing system; romanised means Indian-language text written in Latin letters. Twelve languages get both, among them Hindi, Bengali, Tamil, Telugu, Marathi and Urdu, while ten get native script only, including Assamese, Kashmiri, Sanskrit and Santali. That works out to 34 language-and-script combinations. On Hugging Face the records are tagged by language, script and region, so a reader can tell Hindi in Devanagari from romanised Hindi before downloading anything.

The data does not start from scratch. The team began with English documents from 15 Nemotron-CC-v2 subsets covering web text, question-and-answer material, mathematics and code, then used NVIDIA's open-source NeMo Data Designer to translate, score and repair them. For the twelve dual-script languages the translation model was Google's Gemma-4-31B-it; for the other ten it was Indic Translate, a document-level model from Bodhan AI, a nonprofit incubated at IIT Madras. Every record keeps a link back to its English source, so the text stays auditable. The team also published a Nemotron Bharat translation plugin for NeMo Data Designer, so other builders can reuse the same prompts, windowing and script remapping.

03.The repair step that saved 438 billion tokens

Quality control was strict. Each translation was scored against its source using a rubric inspired by Multidimensional Quality Metrics, covering meaning, fluency, terminology, style and formatting, and only outputs scoring at least 75 out of 100 were kept. Lower-scoring translations were repaired using the assessment feedback and then rescored. That loop alone recovered 438.4 billion tokens that would otherwise have been discarded, a figure already counted in the corpus total.

Handling every language took some improvisation. Longer Santali documents were translated in overlapping windows and stitched back together, while Manipuri was translated into Bengali script and then converted to Meitei Mayek, its native script. The team is candid about the limits too: automated scoring was complemented by sampled human review whose coverage varies by language, and in the twelve-language workflow the same model that translated also judged the result. NVIDIA says the corpus is meant to sit alongside naturally written Indian-language text rather than replace it, and that translation does not by itself capture local knowledge and culture.

04.Proof it helps: Nemotron 3.5 Super VL

The corpus is already feeding a model. Continued pretraining for NVIDIA's Nemotron 3.5 Super VL used a 100-billion-token budget, of which 10 billion tokens were a mix of synthetic and natural Indic text. Scores rose on all five reported Indic benchmarks, namely MILU, Indic MMLU-Pro, Indic ARC-Challenge, Indic ARC-Easy and Indic GSM8K, with gains of 4.98 to 9.82 percentage points, according to the technical report.

NVIDIA is careful to note that the comparison covers a training stage blending synthetic and natural Indic data rather than isolating the new corpus, and that instruction-following on IFEval 1.5 dipped 2.04 points. The model did improve across all four dimensions of IndiVibe, Sarvam AI's open benchmark of assistant-style prompts in Indian languages, covering usefulness, fluency, language and script fidelity, and appropriate verbosity. Still, the direction is clear: more and better Indic data moves the needle on tasks in Indian languages.

05.New benchmarks, and what it means for India

Alongside the data, AI4Bharat released Indic-LLM-Eval-Suite, an open set of offline benchmarks for judging models across eight capability areas including general chat, instruction following, mathematics, STEM, code and translation. The suite pulls together existing Indic benchmarks and refreshed versions such as MILU-1.5 and IndicIFEval-1.5, and is wired into NVIDIA's NeMo-Gym so scores share a common interface. After testing several open-weight Indic and multilingual models, the lab found that no single model performs best across all tasks, a useful reality check for anyone choosing a model.

For Indian startups, students and government services, the practical effect is lower friction. Teams can fine-tune on a large, openly documented Indic corpus instead of assembling data from scratch, and compare results on shared benchmarks rather than on marketing claims. The corpus is releasing soon, and NVIDIA says it plans to build on it in future Nemotron models.