Azure / Azure/azure-sdk-for-python

OSError: Invalid argument when writing to files like meta.json or pf_version_check.json during local Azure AI Evaluation on Windows

Offen
#41,913 4 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
AI Client customer-reported needs-team-attention question Service Attention
Vorherrschende Sprache
Python
Sterne
5.6k
Forks
3.4k
Ø Merge
2 T. 2 Std.
Gemergte PRs (30 T.)
213

Beschreibung

# Bug Report: Azure AI Evaluation OSError on Windows

- **Package Name**: azure-ai-evaluation
- **Package Version**: 1.9.0
- **Operating System**: Windows
- **Python Version**: 3.13.3

## Describe the bug

When running assistant evaluation locally with the Azure AI Evaluation framework on Windows, the process fails with an `OSError: Invalid argument` when attempting to write to files such as `meta.json` or `.promptflow/pf_version_check.json`. This error appears to be related to how Promptflow handles file paths on Windows.

It does not happen consistently. Sometimes, deleting the whole `/.promptflow/` folder helps.

Error stack trace:

```text
OSError: [Errno 22] Invalid argument: '\\.promptflow\\.runs\\azure_ai_evaluation_evaluators_assistant_target_\\meta.json'
```

_This also occurs for `.promptflow/pf_version_check.json`._

Full stack trace excerpt:

```text
Traceback (most recent call last):
...
File "/.venv/Lib/site-packages/promptflow/_sdk/operations/_local_storage_operations.py", line 248, in _dump_meta_file
json_dump({"batch_size": LOCAL_STORAGE_BATCH_SIZE}, self._meta_path)
File "/.venv/Lib/site-packages/promptflow/_sdk/_utilities/general_utils.py", line 872, in json_dump
with write_open(file) as f:
OSError: [Errno 22] Invalid argument: '\\.promptflow\\.runs\\azure_ai_evaluation_evaluators_assistant_target_\\meta.json'
...
```

## To Reproduce

Steps to reproduce the behavior:

1. Set up an evaluation script using the Azure AI Evaluation framework with a local target. Example:
```python
from azure.ai.evaluation import evaluate

result = evaluate(
data=dataset_path,
target=assistant_target,
evaluators=evaluators,
evaluator_config={
"default": {
"column_mapping": {
"query": "${data.query}",
"response": "${target.response}",
"ground_truth": "${data.ground_truth}",
"context": "${target.context}",
}
}
},
output_path=output_path,
azure_ai_project=azure_ai_project,
)
```

3. Run the evaluation on Windows.
4. Occassionally observe the OSError with "Invalid argument" when trying to write to `meta.json` or `.promptflow/pf_version_check.json` files (or sometimes `.promptflow/pf.yaml`)

## Expected behavior

The evaluation script should run successfully without file system errors.

## Additional context

- The error does not occur consistently; sometimes, it helps to first delete the `\.promptflow` folder when running the evaluation script.
- The files in question appear to exist, but the error suggests a problem writing to them.
- This may be related to how Promptflow handles file paths on Windows.

Attempted solutions:
- Changing the `PF_HOME_DIR` to be within another folder
- Setting `PF_WORKER_COUNT` to 1

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Beginne damit, den Windows-Fehler anhand der lokalen Evaluierungsschritte zu reproduzieren, und untersuche promptflow/_sdk/operations/_local_storage_operations.py, insbesondere _dump_meta_file. Verfolge anschließend json_dump und write_open in promptflow/_sdk/_utilities/general_utils.py für die betroffenen meta.json- und .promptflow-Dateien. Als erledigt gilt die Aufgabe, wenn die Evaluierung unter Windows ohne OSError: [Errno 22] abgeschlossen wird.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
python
Bereich
tooling
Issue-Typ
Bug
Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Aktivitätsstatus
Veraltet
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
38/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.