elastic / elastic/elastic-evals-sdk-python
[kbn-evals] Tutorial-grade onboarding example
- Dominant language
- Python
- Stars
- 2
- Forks
- 0
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 18
Description
### Summary
The current example runs a command and ends. It doesn't explain what was created in Kibana, what each score means, or what to look for. A user finishing the example doesn't know what they just did.
### What we want
An example that builds up from scratch: define a dataset, write a task, add evaluators, run it, then walk the user through what appeared in Kibana.
Reference bar: the [DSPy "Program, don't prompt" tutorial](https://dspy.ai/getting-started/program-dont-prompt/) (start small, add one concept at a time, run it, explain the result, move on). The [elasticsearch-labs quick-start notebook](https://colab.research.google.com/github/elastic/elasticsearch-labs/blob/main/notebooks/search/00-quick-start.ipynb) shows the credential-via-env-var pattern to follow.
### Format
Google Colab notebook with an "Open in Colab" badge in the repo. Users run it without installing anything locally. All connection details come from environment variables set in the first cell.
### Connectivity options
The notebook should work with both setups, selected by which env vars the user sets:
**Option A: Elastic Cloud (recommended for Colab)**
A free trial at cloud.elastic.co gives a full Elasticsearch + Kibana stack. The user sets `ELASTIC_CLOUD_URL` and `ELASTIC_API_KEY`. No local installation required.
**Option B: Local stack**
The user runs Elasticsearch and Kibana locally and passes `KIBANA_URL` and `ELASTIC_API_KEY`. The notebook connects to their local stack. No Colab-side setup is needed.
The first cell of the notebook should detect which env vars are present and print a clear message if neither set is found.
### Done when
- [ ] A Colab notebook guides the user from an empty file to a completed run with results in Kibana
- [ ] Each section explains what it does and what to look for after running it
- [ ] The notebook works with Elastic Cloud credentials and with local stack credentials
- [ ] An "Open in Colab" badge appears in the examples README
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.