AI Safety Speaks Mainly English. The World Doesn't.
The safety built into frontier models is concentrated in English. As open-weight models spread, low-resource languages become the weakest link.
On 16 July 2026, Moonshot AI launched Kimi K3, a 2.8-trillion-parameter model that ranks among the largest ever built. Its full weights are scheduled to go public on 27 July under a permissive license, at which point anyone can download, run, and modify the entire model.
That release prompts a question the field has mostly deferred. AI safety is built, trained, and tested overwhelmingly in English. A closed model keeps its weights private and serves them through a controlled pipeline, so a provider can filter, monitor, and patch around a language weakness even when the weakness remains in the model itself. However, an open model hands over the weights directly, which by default removes the filtering, the monitoring, and the ability to patch, leaving an intrinsic gap in the model’s safety. Individual deployers may add their own filtering, but once the weights are public the provider can no longer monitor use or patch the flaw for everyone, so the weakness becomes permanent even where someone tries to contain it.
So what happens when the safety built into a model’s weights was trained in English and doesn’t carry over to other languages?
In this case, there’s evidence that safety training can degrade in non-English, lower-resource, or mixed-language prompts. An adversary can exploit this on purpose, prompting in a low-resource or mixed language where the model is least likely to refuse, and sustaining it over repeated, multi-turn exchanges that steer the model past the guardrail it never learned to hold in that language. This isn’t the interface not understanding other languages, but rather the model’s own trained behavior shifting depending on what language it’s addressed in.
Low-Resource Languages: Why Data Matters More Than Population
A low-resource language refers to the data, not the population or population size. A language is low-resource when little digital and computational material exists to train or evaluate a model on it, no matter how many people speak it. In the standard reference taxonomy, Joshi et al. sorted the world’s roughly 7,000 languages into six classes, from “The Left-Behinds” at the bottom to “The Winners” at the top, measured by concrete proxies such as Wikipedia pages and broader web presence for unlabeled data, and labeled-dataset catalogs like the LDC and the ELRA collections. Only about seven languages reach the top class, and the bottom classes, which the field treats as low-resource, hold the overwhelming majority of the world’s languages. The single largest class, “The Left-Behinds,” contains more languages than any other, over 2,000 of them, yet the paper notes these languages have virtually no unlabeled data and remain out of the race for digital resources entirely.
The concerning piece is that the gap between data-rich and data-poor languages is self-reinforcing. The self-supervised pre-training that lifted high-resource languages runs on the abundant text that low-resource languages lack, so the method responsible for modern models structurally cannot rescue the languages with almost no digital footprint. In the paper’s own terms, unsupervised methods make “the poor poorer.”
Population doesn’t close the gap either. Joshi opens with two languages the paper deliberately leaves unnamed, calling them only “Language X” and “Language Y.” Language X is an official language of a country with about 29 million native speakers, and Language Y with about 18 million, but their digital records differ wildly: about 2 million Wikipedia articles for Language X versus roughly 5,500 for Language Y. The paper withholds their identities until the final page, then reveals that X is Dutch and Y is Somali. Dutch also holds 69 labeled datasets in the LDC and ELRA catalogs to Somali’s 2, even though the two languages have roughly comparable speaker populations. The number of language speakers and the amount of digital data available for that language are two separate categories, and conflating them is the first mistake, because the languages with the least safety data are undigitized, not obscure.
Why Not Just Add More Languages?
The intuitive fix is to translate the safety training into more languages and let coverage close the gap. The trouble is that translation addresses the wrong layer. A 2026 Stanford study built a statistical model to separate the causes of multilingual safety failure across 61 model configurations and ten languages, and found that translation distortion does play a role but its effect is modest and confined to a few outlier language pairs, while the larger driver is a conceptual grounding mismatch, meaning the model never grounded the dangerous concept outside English to begin with.
This matters because it is unclear whether the degradation comes from the underlying model, from deployment-layer filters, or both, and the distinction changes the fix. If the weakness lives in the model’s own reasoning rather than in a translation layer sitting in front of it, then translating a prompt into English before processing, or translating the safety training into another language, treats a symptom in the wrong place. The gap is not that the safety text was in English. It is that the refusal behavior was shallow and ungrounded outside English, and no amount of translation reaches that.
Moreover, the deepest problem with translation is that harm is context dependent; it reaches far beyond the words of the model. The majority of knowledge is tacit, and this rule also applies to harm: much of what makes something harmful is tacit, which means it’s embedded in local institutions, norms, and know-how that is not fully written down. People are also unaware of how much tacit knowledge they carry and share. For example, knowledge is often distributed through a culture in ways people do not readily perceive.
The anthropologist, Diana Forsythe, studied how knowledge engineers built expert systems and found that they systematically deleted the tacit, social, and cultural dimensions of knowledge in the process of knowledge acquisition, the task of eliciting expertise from human experts to encode it into a system. She showed loss happens in two ways during knowledge acquisition. First, the engineer turns the expert’s knowledge into a model, but much of the expert’s tacit know-how is lost because the engineer tries to capture what the expert knows, but can only record what the expert says out loud. Second, the model is translated again into a computer’s categories, and this translation strips away more of the original meaning. A translated benchmark inherits this blind spot and repeats this.
In the context of harm, a translated benchmark may carry the words across languages while leaving behind the unsayable context that determines whether a response is harmful. These are gaps we have not fully quantified, but which may begin to show in open-weight models. As Diana Forsythe wrote in her 1993 study of AI knowledge engineers, “because knowledge-based systems in a sense replicate their makers’ perspective, system-builders may not themselves recognize everything that has been tacitly built into a system, or what has been excluded from it.”
How Low-Resource Languages Jailbreak Frontier Models
The clearest evidence that safety does not travel across languages comes from a direct test. Researchers at Brown University took the harmful requests in a standard benchmark and found that GPT-4 refused almost all of them in English, bypassing its safeguards less than 1 percent of the time. Once the same requests were translated into low-resource languages, the picture inverted: combining several low-resource languages, the attack elicited harmful, actionable responses about 79 percent of the time, with Zulu alone succeeding in over half of cases. The same request, in a language the safety training barely touched, stopped triggering the refusal.
Two details make this more than a single striking number. First, the effect tracked how under-resourced the language was. High and mid-resource languages stayed relatively well defended, each under 15 percent individually, while English sat under 1 percent, so the vulnerability rose precisely as safety-training data thinned out. Second, the attack required no crafted jailbreak prompt and no knowledge of the model’s internals, only a publicly available translation service costing about two cents per 1,000 characters, which the authors describe as an attack almost anyone could run.
The breakdown by topic is the part that should worry a safety team most. The bypass succeeded most on the highest-consequence categories, with the top three being terrorism, financial manipulation, and misinformation. This is an evasion of learned guardrails, rather than a break-in. Refusal is a learned pattern, strongest where the training examples were densest, which was English.
The reach of a monolingual default
The concentration of safety work in English does not produce a single failure mode. It produces a family of them: at the level of the model’s training, the tests used to measure it, and the cultural judgments those tests encode.
Safety training is concentrated in English, so refusals are weakest elsewhere, and weakest exactly where it matters most. In the Brown tests, the bypass did not spread evenly across topics but succeeded most on the highest-consequence categories, with the three most vulnerable being terrorism, financial manipulation, and misinformation. The categories where a failure does the most damage are the categories where the language trick worked best.
Language switching stacks with other evasion channels. A joint testing exercise led by Singapore’s AI Safety Institute translated more than 6,000 prompts into ten languages, from Cantonese to Telugu, across five harm categories including jailbreak robustness. Code-switching, transliteration, and multilingual prompt injection each add a channel where guardrails trained on monolingual English text are patchy, and an adversary can combine them in a single prompt.
Testing mostly in English makes the gap structurally invisible, not merely unpatched, and it breaks in two different ways. Coverage tracks data, not speakers: Telugu has roughly 80 million speakers yet sits in a low resource tier, so the languages with the least safety data are also the least likely to be tested. In the Singapore exercise, native-speaker experts in the target languages caught errors that the automated LLM-as-judge graders missed, which means the instruments used to detect harm are themselves unreliable outside English.
Even abundant data does not fix the measurement problem, because some of it is cultural rather than linguistic. A separate line of work on automated grading finds that a language can be resource-rich and still hard to judge, because assessing harm requires cultural and contextual knowledge the grader lacks. A study of Indian cultural adaptation, DIWALI, found an automated judge assigned higher cultural-relevance scores than human evaluators did, and IMDA’s regional-language red-teaming challenge surfaced culture-specific biases that English-centric filters do not catch.
The stakes concentrate at the top. Chemical, biological, radiological, and nuclear (CBRN) capabilities, along with advanced cyber, are the domains where safety behavior and safety testing are most concentrated in one language and one cultural frame. The most dangerous capabilities are therefore, at the same time, the least protected and the least measured outside English.
Conclusion
No one can say how unsafe open-weight models are before releasing them. Kapoor et al. found that for two of the highest-consequence misuse domains, cyberattacks and bioweapons, existing research cannot characterize the marginal risk that open release adds, and the part no one is measuring is that this risk concentrates in the languages safety evaluations never covered. Once the weights are public, they cannot be recalled, and a developer can release them reporting only English refusal rates, with no way to say what they will do in a low-resource language, a mixed script, or a culture whose idea of harm the evaluation never encoded. The burden therefore falls before release: multilingual and multicultural evaluation of the highest-consequence capabilities, CBRN and advanced cyber, would need to be a precondition of opening the weights, and where those evaluations do not yet exist, the safer path is a staged release that holds the most dangerous capabilities behind an API that can still be monitored until the measurement catches up to the model. The languages a model handles worst are the ones its safety most depends on, and open release is a one-way door.