elastic / elastic/elastic-evals-sdk-python

[kbn-evals] KibanaInferenceClient bypasses shared retry and error stack

Open
#38 0 comments 0 reactions 0 assignees View on GitHub
kbn-evals Team:nightshift-context-and-research
Dominant language
Python
Stars
2
Forks
0
Avg merge
1d 13h
Merged PRs (30d)
19

Description

### Summary

PR #28 added shared helpers for retry logic, error handling, and request headers. Four of five Kibana API clients use them. `KibanaInferenceClient` does not: it has its own copies of everything, including error-body parsing done twice inside the same file. The `prompt()` method also has no retry protection at all.

### Problem

- `inference/client.py:108-115`: retry predicate duplicated from shared stack
- `inference/client.py:142-147`: retry decorator duplicated from shared stack (applied only to `chat_completion`, not `prompt`)
- `inference/client.py:184-193` and `:237-246`: error-body parsing done twice in one file
- `KibanaInferenceError` does not inherit `KibanaAPIError`
- Trace context headers are propagated on inference calls but not on other Kibana API calls

### Fix

Replace duplicated code with the shared stack. Make `KibanaInferenceError` inherit `KibanaAPIError`. Add `**propagated_headers()` to `build_kibana_headers()` so all clients propagate trace context.

### Done when

- [ ] `KibanaInferenceClient` uses `retry_kibana_api_call`, `parse_error_body`, and `raise_kibana_error`
- [ ] Both `chat_completion` and `prompt` have retry protection
- [ ] `KibanaInferenceError` inherits `KibanaAPIError`
- [ ] Error-body parsing appears once in the file
- [ ] All Kibana API clients include W3C trace context headers

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in inference/client.py at the duplicated retry and error-body parsing sections, then compare them with the shared retry_kibana_api_call, parse_error_body, and raise_kibana_error helpers. Inspect build_kibana_headers and the chat_completion and prompt entry points. Done means both methods use retry protection, error handling is shared and defined once, KibanaInferenceError inherits KibanaAPIError, and all Kibana API clients propagate trace context headers.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.