elastic / elastic/elastic-evals-sdk-python
[kbn-evals] Examples should not ship in the installed wheel
- Dominant language
- Python
- Stars
- 2
- Forks
- 0
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 18
Description
### Summary
`pyproject.toml` includes `examples/` in the wheel and pre-registers all three demo suites as entry-point plugins. Every `pip install elastic-evals` ships demo code to the user and shows three suites they didn't create in `elastic-evals list`.
### Problem
- `pyproject.toml:46-47`: `examples` listed in wheel packages
- `pyproject.toml:37-40`: three example suites registered as `elastic_evals.suites` entry points
- `examples` as a top-level module name can collide with other packages
- The three entry-point IDs use inconsistent naming patterns within kebab-case (`agent-builder`, `claude-code-eval`, `chatbot-rag-eval-example`)
### Fix
- Remove `examples` from the wheel packages list
- Remove example entry-point registrations
- Document the plugin entry-point contract so users can register their own suites
### Done when
- [ ] `pip` or `uv` installs no example code
- [ ] `elastic-evals list` on a clean install shows no pre-registered suites
- [ ] The plugin contract is documented with a short example
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with pyproject.toml lines 37-47 to inspect the wheel package list and elastic_evals.suites entry points. Verify a clean pip or uv install contains no examples, confirm elastic-evals list shows no pre-registered suites, and document the plugin entry-point contract with a short example.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, cli, documentation
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 76/100