Azure / Azure/azure-sdk-for-python
`evaluate()` crashes writing results when input data has a `timestamp` (or other date-like) column — `Object of type Timestamp is not JSON serializable`
- Dominant language
- Python
- Stars
- 5.6k
- Forks
- 3.4k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 193
Description
- **Package Name**: azure-ai-evaluation
- **Package Version**: 1.17.0
- **Operating System**: Windows-10-10.0.26200-SP0
- **Python Version**: 3.11.15
**Describe the bug**
`azure.ai.evaluation.evaluate()` raises `EvaluationException (InternalError) Object of type Timestamp is not JSON serializable` when the input `data` JSONL contains a column whose name pandas treats as date-like (e.g. `timestamp`, or any column ending in `_at`/`_time`, or named `date`/`modified`).
Root cause is two SDK behaviors combining:
1. `JSONLDataFileLoader.load` reads input with `pd.read_json(self.filename, lines=True, dtype=object)` (`_evaluate/_utils.py`). `read_json`'s `convert_dates` defaults to `True`, so a string column named `timestamp` is silently coerced to `pandas.Timestamp` values (`dtype=object` does **not** prevent this — the cells become `Timestamp` objects).
2. `_write_output` serializes the result with a bare `json.dump(data_dict, f, ensure_ascii=False)` (`_evaluate/_utils.py`) with no `default=` handler. The leaked `Timestamp` then fails to serialize.
The input column is pass-through data (not consumed by any evaluator), so there's no way to anticipate that naming a column `timestamp` will break result writing.
**To Reproduce**
Steps to reproduce the behavior:
1. Create an input JSONL (`data.jsonl`) with a string `timestamp` column:
```jsonl
{"query": "What is 2+2?", "response": "4", "timestamp": "2026-06-26T17:28:48Z"}
```
2. Run any evaluator with an `output_path`:
```python
from azure.ai.evaluation import evaluate, CoherenceEvaluator
evaluate(
data="data.jsonl",
evaluators={"coherence": CoherenceEvaluator(model_config)},
output_path="results.json",
)
```
3. Evaluation runs, but writing the output file fails with the stack trace below.
**Expected behavior**
`evaluate()` writes results successfully. Pass-through input columns should be preserved as-is (a string `timestamp` should remain a string), or at minimum the result writer should be able to serialize the dtypes the SDK itself produces.
**Screenshots**
N/A — stack trace:
```
File ".../azure/ai/evaluation/_evaluate/_evaluate.py", line 1110, in _evaluate
_write_output(output_path, result)
File ".../azure/ai/evaluation/_evaluate/_utils.py", line 337, in _write_output
json.dump(data_dict, f, ensure_ascii=False)
...
File ".../json/encoder.py", line 180, in default
raise TypeError(f'Object of type {o.__class__.__name__} is not JSON serializable')
TypeError: Object of type Timestamp is not JSON serializable
azure.ai.evaluation._exceptions.EvaluationException: (InternalError) Object of type Timestamp is not JSON serializable
```
**Additional context**
Minimal confirmation using the SDK's own data loader — reproduces the type coercion and the serialization failure without requiring a judge model or network:
```python
import json, tempfile, os
from azure.ai.evaluation._evaluate._utils import JSONLDataFileLoader
p = os.path.join(tempfile.gettempdir(), "probe_data.jsonl")
with open(p, "w", encoding="utf-8") as f:
f.write(json.dumps({"query": "hi", "response": "yo", "timestamp": "2026-06-26T17:28:48Z"}) + "\n")
df = JSONLDataFileLoader(p).load()
print("loaded type:", type(df["timestamp"].iloc[0]).__name__) # Timestamp (the input was a str)
json.dumps(df.to_dict(orient="records")) # TypeError: Object of type Timestamp is not JSON serializable
```
Suggested fix — either (or both):
- **Read-layer (closer to root cause):** pass `convert_dates=False` to `pd.read_json` in `JSONLDataFileLoader.load` so user data columns aren't silently retyped.
- **Write-layer (defensive):** give the `json.dump` in `_write_output` a `default=` handler that serializes datetime-like objects (`isoformat()`) and numpy scalars (`.item()`), so no pandas/numpy dtype can break result writing.
Happy to open a PR if you can confirm the preferred approach.
pandas version: 2.3.3
Contributor guide
Assessment
This issue has not been assessed yet.