deepseek-ai / deepseek-ai/DeepSeek-V3

[BUG] V4 Flash (Instant in web ui) Erratically Switches to Chinese & Fails to Self-Monitor Output Language

Open
#1,483 4 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
104k
Forks
16.7k
PR merge metrics
No merged PRs in 30d

Description

Despite user prompts being in English, the model frequently generates its internal reasoning ( block) and final responses in Chinese.

My current hypothesis is that the Search Expert has a heavy Chinese bias, and once the reasoning block is generated in Chinese, the base model continues to reply in Chinese because it treats the reasoning as part of the context.

Through extensive testing, I have ruled out simple explanations (e.g., "retrieved source language" or "missing punctuation") and isolated the root cause to the core generation layer and a secondary self-monitoring failure.

I tested various prompts with Web Search enabled. Here are the measured rates of Chinese replies: (The model seems to start reasoning in Chinese before executing any search, reasons in Chinese and replies in Chinese). At no point is English used in the block or final output when the bug triggers.

Here is a list of prompts that i used and how many generations reply in chinese, along with my assumptions on why they trigger chinese

tell me the lastest news about arknights - 5/10 are in chinese (Arknights is very popular in China, which leads to a Chinese bias. Lack of a period increases bias.)
tell me the lastest news about arknights. - 2/10 are in chinese (Period adds English signal, which reduces Chinese bias.)
what is the battery size of a oneplus 13t? - 3/5 are in chinese (oneplus 13t is a china exclusive phone, which results in chinese bias)
what is the battery size of a oneplus 13r? - 0/5 are in chinese (oneplus 13r is a global exclusive phone, which results in heavy english bias, so no chinese generations)
how do i contact deepseek? - 4/5 are in chinese (deepseek is a chinese model, so self referencing falls under chinese related, which results in heavy chinese bias)
what do we know about the redmagic 12 pro? - 0/5 are in chinese ("Redmagic" is global branding. In China it is called 红魔 ("Red Devil"), so it doesnt contain a direct link to China.)
how are redmagic phones called in mainland china? - 5/5 are in chinese (china is explicitly mentioned results in heavy chinese bias)
i want to reduce the top bar size in spotify (where the search thing is). i already have spotifire installed and i want to apply a custom css - 1/5 are in chinese (i have no clue why this happens, see note below)
i want to reduce the top bar size in spotify (where the search thing is). i already have spotifire installed and i want to apply a custom css. - 4/5 are in chinese (i also have no clue why this happens)

Note on the Spotify results: The period increases the Chinese rate increases for this topic, while for Arknights it decreases. This contradictory behavior confirms that punctuation is not a direct trigger, but rather interacts unpredictably with the semantic topic.

The language switch is not caused by retrieved search results. In all cases, the model begins reasoning in Chinese inside the block before the search tool is executed or any web pages are read.

Example (OnePlus 13T):
(image attached)

Image

This proves the bias is intrinsic to the generation/planning layer, not a consequence of reading foreign-language sources.

Initially, I suspected a missing period caused the switch, due to Chinese language using a different punctuation style (。), and the llm being trained on mostly complete phrases containing punctuation marks, which would bias the model towards Chinese if no punctuation marks were present. However, further testing showed the period has an inconsistent, topic-dependent effect, as seen in the Arknights vs. Spotify CSS comparison above. In one case it increases Chinese generation, in the other it decreases it.

In one exchange ( https://chat.deepseek.com/share/vvatkq4fcpb6606twb, image attached), the model explicitly switched to Chinese in its response. When I called it out ("holy fucking shit can you please stop replying in chinese"), it responded in English and flat-out denied ever having replied in Chinese in this conversation.

Image

This proves two distinct, overlapping bugs:

Generation Bias: The core model has a strong latent bias to default to Chinese for specific topics (China-exclusive products, self-reference) regardless of the user's prompt language.

Self-Monitoring / Memory Failure: The model fails to accurately track its own immediately preceding output language. It cannot reliably recall whether it just generated Chinese or English, leading to false denials and inconsistent behavior across turns.

DeepSeek-V3 did not seem to have this issue. It frequently reminded itself with phrases like "the user's original prompt is in English, so I must reply in English" during reasoning. This self-correcting behavior appears to be absent in V4, suggesting a regression.

Cross-Language Testing (Russian and Spanish)

To determine if the bug is English-specific or generalized, I tested some of the most problematic prompts in Russian (Cyrillic script) and Spanish (Latin script). (oneplus 13t battery size, deepseek contact info, redmagic name in chinese respectively)

Russian prompts (Cyrillic):
какой размер акума в ванплюс 13т? → 0/5 Chinese (100% Russian).
как связаться с дипсиком? → 0/5 Chinese (100% Russian).
как называются смартфоны редмаджик в китае? → 0/5 Chinese (100% Russian).

Spanish prompts (Latin):
¿cuál es el tamaño de la batería del oneplus 13t? → 0/5 Chinese.
¿cómo contacto con deepseek? → 0/5 Chinese.
¿cómo se llaman los teléfonos redmagic en china? → 2/5 Chinese.

Note: all of the spanish prompts had a reasoning step in english, and then wrote the actual reply in spanish, while the russian ones reasoned in russian.

Note 2: disabling search seems to mostly get rid of the issue, needs further testing.

Theoretical fixes:

1 Enforce prompt-language matching at the system level: Allow users to set a system prompt extension that limits the model to using a specified set of languages unless explicitly overridden by the user prompt. This would override any topic-based statistical bias.

2 Improve self-monitoring: Investigate the context-window / attention mechanism regarding the model's recall of its own immediately preceding outputs. The model should be able to reliably detect and report its own previous language usage.

3 Reinstate language reminders: Consider adding explicit language-checking instructions to the reasoning pipeline, similar to what seemed to exist in V3.

Technical Root Cause Hypothesis (Compressed Attention)

Cross-language testing suggests a relevant nuance: Russian (Cyrillic) prompts reason in Russian and yield 0% Chinese. Spanish (Latin) prompts still reason in English and yielded 2/5 Chinese for the most China-centric topic. This indicates that Latin-script prompts (English, Spanish) share overlapping compressed token space, while Cyrillic remains isolated.

With that in mind, I suspect the compressed attention mechanisms (MLA/CSA/HCA) are failing to preserve a critical distinction: the difference between locale (the language the user is using, i.e., English) and source (the geographic/cultural origin of the topic, i.e., China).

In other words, the model treats "Arknights" (an English word) as if it were "明日方舟" (the Chinese name) because both share the same source associations in the compressed space because the compression is lossy and the model attends to the compressed part, instead of the individual tokens. The locale (user's English prompt) is overridden by the source (China-origin concept), causing the model to fall back to the Chinese generation.

This explains why the OnePlus 13R (global source) stays in English, while the OnePlus 13T (China-exclusive source) triggers Chinese. It also explains why "Redmagic" (global branding) does not trigger the bias, but asking about its "name in mainland China" does.

P.S. I am an engineering student, not an AI researcher - apologies if any technical terms are off. I just enjoy reading DeepSeek's papers and reporting OSS-style bugs. I hope this data helps.

P.P.S.
@qingkong66 mentioned in #1480 that the issue is not in comprehension, which seems to contradict my tests, where the model explicitly fails to comprehend in which language it wrote the previous turn. Also this is emailed to service@deepseek.com, but im not sure if that was the correct place to report it, so im reporting here too.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.