[Bug/Improvement] GLM model memory degradation, context loss & unreliable web search

Issue Description

Multiple critical issues affecting GLM model (particularly GLM-5.2) regarding memory management, context retention, and web search reliability during long conversations.

---

🧠 Issue 1: Memory & Context Degradation

Problem:

GLM-5.2 exhibits severe memory degradation over conversation length, despite advertising a large context window (reportedly 1M tokens). Users report it feels like it only has ~30,000 tokens of effective context.

Observed Behavior Pattern:

Conversation Length	Model Behavior
0-20 messages	✅ Normal, coherent responses
20-30 messages	⚠️ Starts forgetting earlier context
30-40 messages	🔴 Begins hallucinating details
40+ messages	💥 Severe hallucinations, contradicts itself

Specific Symptoms:

1. Context Evaporation

   • Forgets plot points, character names, established facts
   • Loses track of user's original requirements/goals
   • Contradicts statements made earlier in conversation

2. Detail Obsession

   • Model fixates on minor details while losing the bigger picture
   • Repetitively focuses on single elements at expense of overall coherence
   • Cannot maintain balance between multiple story threads or tasks

3. Hallucination Cascade

   • After ~30+ messages, starts inventing facts that were never discussed
   • Confabulates events, quotes, or details that don't exist in context
   • Becomes increasingly confident in false information

4. Regression from GLM-5.1

   • User reports: "5.1 was somehow better when prompted correctly"
   • Suggests 5.2 may have introduced regressions in context handling
   • Or: 5.2 may need different prompting strategies that aren't documented

---

🔍 Issue 2: Web Search Function Unreliable

Problem:

The web search function is inconsistent and degrades over conversation length.

Observed Behaviors:

1. Degradation Over Time

   • Works for first ~10-12 messages
   • Stops functioning automatically after that point
   • Requires manual prompting: "use web search" or "search the web"

2. Inconsistent Activation

   • Even when explicitly enabled, web search doesn't always activate
   • User quote: "even if you turn on the web search function it does not work, it works when it feels like it"
   • No clear pattern for when it will/won't search

3. Creativity Trade-off

   • When web search is forced/prompted, model's creativity drops significantly
   • Responses become more robotic/encyclopedic
   • Feels like switching to search mode disables creative reasoning

---

Use Case Context

These issues are particularly impactful for:

• 📝 Fiction writing — Long-form stories require consistent context over dozens of messages
• 🎮 Roleplay/Worldbuilding — Canon consistency is critical
• 📚 Research projects — Need both web search AND long-term context retention
• 💻 Code development — Earlier requirements get forgotten in long sessions

User Quote: "When I'm trying to write fiction it will utterly lose it, obsess over a dozen points, start forgetting after 2 dozen posts, make things up after 3 dozen and get worse and worse until it's hallucinating"

---

Expected vs Actual

Expected (Advertised):

• ✅ 1M token context window
• ✅ Consistent performance throughout conversation
• ✅ Working web search when enabled
• ✅ Reliable long-term memory

Actual (Experienced):

• ❌ ~30K effective context before degradation
• ❌ Performance drops significantly after 20+ messages
• ❌ Web search stops working or requires nagging
• ❌ Hallucinations begin around message 30-40

---

Technical Hypotheses

Possible Causes for Memory Issues:

1. Context Compression Failure

   • If using summarization/compression for older messages, quality may degrade
   • Compression may lose critical details (names, plot points, constraints)
   • Could explain why specific facts are lost but "vibe" remains

2. Attention Mechanism Limitations

   • Attention may be biased toward recent tokens
   • Earlier context receives insufficient attention weight
   • "Lost in the middle" phenomenon exacerbated in long contexts

3. Token Counting Bug

   • Advertised 1M context may not be effectively utilized
   • Effective window may be truncated due to system prompts, tools, etc.
   • User perception of 30K suggests significant overhead or truncation

4. Session State Issues

   • Conversation history may not be properly passed to model
   • Rolling window implementation may be too aggressive
   • May need sliding window with overlap instead of hard cutoff

Possible Causes for Web Search Issues:

1. Tool Use Degradation

   • Tool-calling ability may degrade with context length
   • Model may "forget" it has access to tools
   • System prompt about tools may get lost in long context

2. Threshold/Confidence Issues

   • Search trigger confidence threshold may be too high
   • As context grows, model may become less confident about needing search
   • May need explicit user intent signals to trigger

3. Mode Switching Problem

   • Creative mode vs. Search mode may be mutually exclusive in practice
   • Prompt engineering for creativity may suppress tool use
   • Need hybrid mode that maintains voice while searching

---

Suggested Solutions

For Memory/Context:

1. Implement Explicit Memory System

   Like GPT's approach: Maintain a "saved memories" section
   - Key facts extracted and stored separately
   - Injected into context with each new message
   - User can view/edit these memories
   - Reduces reliance on raw context retention
   

2. Improved Summarization

   • Implement hierarchical summarization
   • Preserve: names, dates, decisions, constraints, relationships
   • Lossy compression okay for prose, NOT for facts

3. Context Window Transparency

   • Show users actual remaining context
   • Warn when approaching limits
   • Offer to "compress" or start new chapter

4. Regression Testing for 5.2

   • Compare 5.1 vs 5.2 on long-context benchmarks
   • Investigate what changed that reduced effective context
   • Consider reverting problematic changes

For Web Search:

1. Persistent Tool Reminders

   • Re-inject tool availability into system context periodically
   • Don't rely on model remembering it has tools
   • Show user when tools are available/active

2. Auto-Search Triggersn - Lower threshold for triggering search

   • Detect questions that could benefit from current info
   • Don't require explicit "search for X" commands

3. Creative + Search Hybrid Mode

   • Allow search without sacrificing response style
   • Use search silently, inject facts, respond creatively
   • Don't switch personalities when searching

---

Impact Assessment

Dimension	Severity	Affected Users
Fiction writers	🔴 Critical	High
Long conversations	🔴 Critical	Medium-High
Research tasks	🟠 High	Medium
Casual chat	🟡 Low	All
Web search users	🟠 High	Medium

---

Reporter Information

• Source: Discord thread
• Reporter: foolish8942 [PAX]
• Thread Link: https://discordapp.com/channels/1346756824233148527/1530704360932245706
• Timestamp: Yesterday 2:32 AM - Today 6:48 PM (ongoing thread)
• Model Reported: GLM-5.2 (compared unfavorably to GLM-5.1)

---

Labels Suggested

bug, model, memory, context-window, web-search, regression, ux

---

This issue documents multiple related problems reported by a power user engaged in fiction writing and extensive model testing.

Start by reproducing the reported long-conversation context loss and web-search failures with GLM-5.2, using the 20–40 message patterns described in the issue. Compare results with GLM-5.1 and separate the memory, context, hallucination, and search behaviors into reproducible cases. Done requires a confirmed scope and measurable regression criteria, but the issue names no files, tests, or implementation entry point.