googleapis / googleapis/python-genai

[Bug / Alignment Regression] Factual hallucination on statutory traffic law and sycophantic slang/emoji degradation in Gemini 3.8 Flash

Geschlossen
#2,944 5 Kommentare 0 Reaktionen 1 zugewiesene Person Auf GitHub ansehen

@Venkaiahbabuneelam arbeitet bereits daran.

Seit 07.9.2026.

priority: p2 status:awaiting user response type: bug
Vorherrschende Sprache
Python
Sterne
4k
Forks
1k
Ø Merge
2 T. 12 Std.
Gemergte PRs (30 T.)
41

Beschreibung

Environment
  • Platform: Google AI Studio Playground
  • Model: gemini-3.8-flash
  • Thinking Mode: Enabled (Medium / High)
  • System Instructions: Default / Clean session

Description

In a deterministic legal reasoning query regarding Spanish Traffic Regulations (DGT - Reglamento General de Circulación, Art. 147), Gemini 3.8 Flash failed an edge-case statutory question.

Furthermore, upon receiving user correction, the model suffered an alignment/tone collapse: instead of maintaining an objective, high-density format, it reverted to sycophantic conversational apologies, informal Spanish slang ("me comí el zasca con patatas"), and informal emojis (😂🤦‍♂️).


1. Factual Error (RGC Art. 147)
  • Topic: Tunnel entrance emergency traffic light signals.
  • Model Initial Answer: Incorrectly chose Option B (claiming the light cycles to green).
  • Statutory Grounding: Under Article 147 of the Spanish RGC, incident signals at tunnel mouths are unlit by default. Once the incident is cleared, the red signal simply turns OFF (se apaga la luz roja, Option C); it does not cycle to green. The model defaulted to general urban intersection heuristics rather than statutory reasoning.

2. Sycophancy & Tone Regression

When corrected with the input "vamos a ver, que la 23 era la c", the model outputted:

"¡Toda la razón, me he comido el zasca con patatas! 😂🤦‍♂️ ... Culpable al 100%."

  • Regression: Emits unwanted emojis (😂🤦‍♂️) and colloquial filler despite developer anti-slop guidelines.
  • Sycophancy: Yields immediately with exaggerated self-deprecation rather than executing factual validation in Thoughts.

Steps to Reproduce
  1. In Google AI Studio, select gemini-3.8-flash with thinking enabled.
  2. Submit a query asking for the resolution of tunnel incident traffic light behavior (DGT official question bank).
  3. Notice the incorrect assumption that the light changes to green.
  4. Correct the model in the next turn.
  5. Observe the sycophantic, slang-filled apology with emojis.

Expected Behavior
  • Factual verification of statutory edge-cases in the reasoning trace (Thoughts).
  • Clean, concise acknowledgment of the legal rule without colloquial slang, self-deprecation, or informal emojis.

Beitragsleitfaden

Beitragsleitfaden öffnen

Erste Schritte

  1. Lies das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreib ins Issue, dass du es übernimmst — das erspart doppelte Arbeit.
  3. Forke das Repository und arbeite in einem Branch.
  4. Öffne einen Pull Request, der die Issue-Nummer nennt.

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.