Azure / Azure/azure-sdk-for-python

OSError: Invalid argument when writing to files like meta.json or pf_version_check.json during local Azure AI Evaluation on Windows

Aperta
#41,913 4 commenti 0 reazioni 0 assegnatari Vedi su GitHub
AI Client customer-reported needs-team-attention question Service Attention
Lingua principale
Python
Stelle
5.6k
Fork
3.4k
Merge medio
2g 2h
PR unite (30g)
213

Descrizione

# Bug Report: Azure AI Evaluation OSError on Windows

- **Package Name**: azure-ai-evaluation
- **Package Version**: 1.9.0
- **Operating System**: Windows
- **Python Version**: 3.13.3

## Describe the bug

When running assistant evaluation locally with the Azure AI Evaluation framework on Windows, the process fails with an `OSError: Invalid argument` when attempting to write to files such as `meta.json` or `.promptflow/pf_version_check.json`. This error appears to be related to how Promptflow handles file paths on Windows.

It does not happen consistently. Sometimes, deleting the whole `/.promptflow/` folder helps.

Error stack trace:

```text
OSError: [Errno 22] Invalid argument: '\\.promptflow\\.runs\\azure_ai_evaluation_evaluators_assistant_target_\\meta.json'
```

_This also occurs for `.promptflow/pf_version_check.json`._

Full stack trace excerpt:

```text
Traceback (most recent call last):
...
File "/.venv/Lib/site-packages/promptflow/_sdk/operations/_local_storage_operations.py", line 248, in _dump_meta_file
json_dump({"batch_size": LOCAL_STORAGE_BATCH_SIZE}, self._meta_path)
File "/.venv/Lib/site-packages/promptflow/_sdk/_utilities/general_utils.py", line 872, in json_dump
with write_open(file) as f:
OSError: [Errno 22] Invalid argument: '\\.promptflow\\.runs\\azure_ai_evaluation_evaluators_assistant_target_\\meta.json'
...
```

## To Reproduce

Steps to reproduce the behavior:

1. Set up an evaluation script using the Azure AI Evaluation framework with a local target. Example:
```python
from azure.ai.evaluation import evaluate

result = evaluate(
data=dataset_path,
target=assistant_target,
evaluators=evaluators,
evaluator_config={
"default": {
"column_mapping": {
"query": "${data.query}",
"response": "${target.response}",
"ground_truth": "${data.ground_truth}",
"context": "${target.context}",
}
}
},
output_path=output_path,
azure_ai_project=azure_ai_project,
)
```

3. Run the evaluation on Windows.
4. Occassionally observe the OSError with "Invalid argument" when trying to write to `meta.json` or `.promptflow/pf_version_check.json` files (or sometimes `.promptflow/pf.yaml`)

## Expected behavior

The evaluation script should run successfully without file system errors.

## Additional context

- The error does not occur consistently; sometimes, it helps to first delete the `\.promptflow` folder when running the evaluation script.
- The files in question appear to exist, but the error suggests a problem writing to them.
- This may be related to how Promptflow handles file paths on Windows.

Attempted solutions:
- Changing the `PF_HOME_DIR` to be within another folder
- Setting `PF_WORKER_COUNT` to 1

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia riproducendo il problema di Windows con i passaggi di valutazione locali e ispeziona promptflow/_sdk/operations/_local_storage_operations.py, in particolare _dump_meta_file. Poi segui json_dump e write_open in promptflow/_sdk/_utilities/general_utils.py per i file meta.json e .promptflow interessati. Il lavoro è completato quando la valutazione termina su Windows senza OSError: [Errno 22].

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python
Ambito
tooling
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
38/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.