aws-samples / aws-samples/sample-ai-possibilities

teamChat commands are lost during state summarization and never reach Agent Context

Open
#146 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
40
Forks
36
Avg merge
5h 26m
Merged PRs (30d)
1

Description

## Summary

Coach commands sent from the Player Portal through `gameState.teamChat` are received by the Agent service but are not available to the LLM during decision making.

Example commands:

```
defend
press
shoot
```

Currently, these commands do not influence player behavior.

---

## Expected Behavior

Coach instructions should be included in the player agent context:

```
GameState

Context Construction

Agent Context

LLM Reasoning

Player Action
```

The agent should be able to consider tactical instructions together with the current match state.

---

## Actual Behavior

Current execution pipeline:

```
Game Engine

gameState payload

agent_base.py / gateway invoke

summarize_state()

Agent(state_summary)

LLM reasoning
```

The Agent does not receive the complete raw `gameState`.

It only receives:

```
state_summary
```

Only fields extracted by `summarize_state()` are visible to the LLM.

Current field mapping:

```
gameState

├── ball ✅ extracted
├── players ✅ extracted
├── score ✅ extracted
├── gameTime ✅ extracted
├── playMode ✅ extracted
└── teamChat ❌ not extracted
```

---

## Runtime Evidence

Logs captured before and after `summarize_state()` show that `teamChat` exists in the raw game state but disappears during context construction.

### Before summarize_state()

Input `gameState`:

```json
{
"teamChat": [
{
"senderTeamCode": "home",
"message": "defensive",
"timestamp": 0
}
]
}
```

The coach instruction exists in the incoming game state.

---

### After summarize_state()

Generated agent context:

```
Time: 11s
Score: 1-0
Team: 0 (HOME)
PlayMode: RESETTING
Ball: (7.0, 0.4) held by free
```

The `teamChat` field is missing.

This confirms the issue occurs during:

```
gameState -> Agent Context
```

conversion.

---

As a result, the coach command is not included in the LLM context during decision making.

Contributor guide

Open the contributing guide

Research direction

Start in agent_base.py at summarize_state() and trace the gateway invoke path that constructs Agent(state_summary). Compare the raw gameState mapping with the generated context, focusing on teamChat. Done means coach commands are present in Agent Context and available during LLM decision making.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.