Allow configuring languages for the multilingual tokenizer

Is your feature request related to a problem? Please describe.
Thank you for the multilingual tokenizer work. I would like to use it for Japanese search, but short Japanese queries are often not detected as Japanese.

This matters because many real search queries are short keywords, often separated by spaces. In those cases, the current language detection seems to fail often enough that Japanese full-text search becomes unreliable.

For example, I would expect both of these queries to be treated as Japanese:

• 日経平均株価
• 日経平均株価が

But they are tokenized very differently:

• 日経平均株価 -> ["日", "經", "平均", "株", "價"] (NG, Chinese-like tokenization)
• 日経平均株価が -> ["日経", "平均", "株価", "が"] (OK, Japanese tokenization)

Mixed Japanese/English queries are also affected:

• nvidia 時価総額 -> ["nvidia", "時", "價", "總", "額"] (NG, not Japanese tokenization; some characters are also normalized to different Unicode code points, for example 価 -> 價, 総 -> 總)
• expected Japanese-aware tokenization: ["nvidia", "時価", "総額"]

At least for Japanese, the impact is large because short keyword queries are very common in search UX.

Describe the solution you'd like
My main request is to allow users to configure which languages are considered by the multilingual tokenizer, so language detection can be constrained for a given field.

For example, an API shape similar to this might help:

{
  "field_schema": {
    "type": "text",
    "tokenizer": "multilingual",
    "languages": ["japanese", "english"]
  }
}

With something like this, short queries could be detected only within the configured languages, instead of being classified as an unrelated language such as Chinese.

For my use case, this would already be enough. I do not necessarily need a perfect general-purpose language detector. I mainly need a way to prevent short Japanese queries from falling into the wrong language path.

Describe alternatives you've considered

• Replacing the current language detection with another implementation or library that performs better on short queries

However, that seems to have a wider implementation impact.

By comparison, allowing users to constrain the detected languages would likely reduce unintended language classification a lot for this use case, with a smaller change.

Additional context
This seems to be a language detection problem before tokenization, not only a tokenization problem itself.

As a reference point, the same example queries are detected as Japanese in the Chrome Translation / Language Detection API playground:

• https://chrome.dev/web-ai-demos/translation-language-detection-api-playground/

No implementation files or tests are named. Start by locating the multilingual tokenizer entry point and its language-detection configuration, then reproduce the Japanese and mixed Japanese/English examples from the issue. Done means a field can restrict detection to configured languages and the examples no longer fall into Chinese-like tokenization.