apache / apache/burr

Burr Serving Framework

Open
#304 3 comments 1 reaction 0 assignees View on GitHub
area/core kind/feature priority/low
Dominant language
Python
Stars
2.5k
Forks
195
Avg merge
6d 10h
Merged PRs (30d)
17

Description

**Is your feature request related to a problem? Please describe.**
We have some examples of serving in FastAPI. They all follow the same pattern. We should be able to automate this (to some extent):
- [streaming chatbot](https://blog.dagworks.io/p/streaming-chatbot-with-burr-fastapi)
- [email assistant](https://towardsdatascience.com/building-an-email-assistant-application-with-burr-324bc34c547d)

**Describe the solution you'd like**
A few thoughts:

In FastAPI:

```python
burr_app = Application()....build()
app = FastAPI()
burr_fastapi.post("/api/v0/...", app, burr_app, halt_after=..., halt_before=..., ..., response_model=CustomResponseModel)
burr_fastapi.post_streaming("/api/v0/...", app, burr_app, halt_after=..., halt_before=..., ..., response_model=CustomResponseModel)
```

Then `CustomResponseModel` has to be instantiatable from state dict (E.G. be a subset/have the right defaults).

Could also decorate functions -- these inject the results in

```python
burr_app = Application()....build()
app = FastAPI()
@burr_fastapi.post("/api/v0/...", app, burr_app, halt_after=..., halt_before=..., ...)
def my_burr_endpoint(burr_state, burr_action, burr_result) -> MyResponse:
return MyResponse(...)
```

Note that we could also derive it if we set up the fields we expose in state + add #139 -- this is something we'd want to do later, but...

```python
burr_app = ...
app = FastAPI()
burr_fastapi.post("/api/v0/...", app, burr_app, halt_after=..., halt_before=..., ...)
```

This could automatically derive the response model based on types statically inferred + halt_before/halt_after, although it's a little complicated...

**Describe alternatives you've considered**
Can do custom like we do now.

**Additional context**
As asked on discord -- this is part 1

Contributor guide

Open the contributing guide

Research direction

Review the linked FastAPI serving examples and the current custom serving pattern before narrowing the scope. Compare the proposed burr_fastapi.post and post_streaming entry points, including halt controls and response-model/state behavior, and account for the relationship to follow-up #139. Done should mean an agreed first-part API with documented acceptance tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
fastapi, python
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.