Lightning-AI / Lightning-AI/LitServe
Pass headers to litapi
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.9k
- Forks
- 304
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 6
Description
## 🚀 Feature
Being able to access headers in the litapi stages (decode, predict, etc.).
### Motivation
- Tracing - user agents, region and similar in distributed logging across stages (decode, predict, etc.)
- Model tweaks (different model, different thresholds, etc.) based on implicit headers (like user agent).
### Pitch
There is benefit to accessing headers natively in litapi that would provide more flexibility in the model inference and execution since headers are being send implicitly by clients and in some cases they can differ the way predict should be executed.
### Alternatives
Currently, we need to repack the request body if we want to add header value to the request object ending up in the litapi.
Or, caller must pass them in body.
### Additional context
https://github.com/Lightning-AI/LitServe/issues/566
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file or test is named. Start by tracing how requests enter litapi and reach the decode and predict stages, then identify where headers are currently discarded. Done means stage code can access the incoming headers without requiring callers to repack them into the request body, with behavior covered by relevant tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100