Ecologits queries direclty codecarbon when installed in inference server
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.9k
- Forks
- 323
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 12
Description
Is your feature request related to a problem? Please describe.
This is more a long shot proposition than a real problem. To extend ecologits usage on remote APIs that are using codecarbon at inference time, a direct connexion could be established between ecologits & codecarbon clients, to exchange data measured at request level.
Describe the solution you'd like
The ideal solution would detect the presence of codecarbon on the remote API, then open a direct connexion with the codecarbon client & gets data for each request, in the same way it would have processed with the proxy metrics of the Ecologits client.
In order to complete this, first an integration in the remote APIs is necessary. The current work ongoing on vLLM could help demonstrate the proof of concept
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file or test is named. Start by reviewing the current vLLM integration and how Ecologits proxy metrics are processed, then identify the remote API integration points for a CodeCarbon client. Done would mean a demonstrated integration that detects CodeCarbon and exchanges measurements for each request.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, api, backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100