MagicStack / MagicStack/httptools

[Bug] Request body lost when Upgrade: h2c + Transfer-Encoding: chunked is used

未关闭
#124 0 条评论 2 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

主要语言
Python
星标
1.3k
派生
107
PR 合并指标
30 天内没有已合并 PR

描述

Overview

When sending a POST request from a Java RestClient (Spring Boot 3.2+, Java 21) to a FastAPI backend running on Uvicorn + httptools, we encountered a strange issue where the request body was missing.

The request looked like this:

POST /endpoint HTTP/1.1
Host: my-api.com
Upgrade: h2c
Connection: Upgrade, HTTP2-Settings
Transfer-Encoding: chunked
Content-Type: application/json

3\r\nabc\r\n0\r\n\r\n

On the server side, Uvicorn logs showed:

  • Unsupported upgrade request
  • No request body
  • Invalid HTTP request received

But when we routed the same request through ngrok or used RestTemplate instead of RestClient, it worked fine.


🔍 Root Cause

After analyzing Uvicorn’s httptools_impl.py and httptools parser behavior, we found this:

  • Upgrade: h2c is ignored by Uvicorn (as expected).
  • But internally, httptools still enters the upgrade state.
  • Since the upgrade is ignored and the parser is not reset, no body is parsed.
  • This violates RFC 7230 §6.7, which allows the server to ignore upgrades and proceed normally.

Proposed Fix

Patch parser.pyx to resume HTTP/1.1 parsing after upgrade is ignored:

cdef int cb_on_headers_complete(cparser.llhttp_t* parser) except -1:
    cdef HttpParser pyparser = <HttpParser>parser.data
    try:
        if parser.upgrade and not pyparser._should_upgrade():
            cparser.llhttp_resume_after_upgrade(parser)
        pyparser._on_headers_complete()
    except BaseException as ex:
        pyparser._last_error = ex
        return -1
    return 0

Also expose this from Python:

def resume_after_upgrade(self):
    httptools.llhttp_resume_after_upgrade(self.cparser)

Then frameworks like Uvicorn can call it in:

def on_headers_complete(self):
    if self.upgrade and self.upgrade.lower() != b"websocket":
        self.parser.resume_after_upgrade()

Reproducible Test

def test_chunked_body_with_ignored_upgrade():
    headers = {
        "Upgrade": "h2c",
        "Connection": "Upgrade",
        "Transfer-Encoding": "chunked"
    }
    body = b"4\r\ntest\r\n0\r\n\r\n"
    request = b"POST / HTTP/1.1\r\n" + headers_to_bytes(headers) + b"\r\n" + body

    parser = HttpRequestParser(TestProtocol())
    parser.feed_data(request)

    assert protocol.body == b"test"

Why it matters

This is RFC-compliant behavior that should be supported.

RestClient in Java 21+ sends Upgrade: h2c by default.

Any server not resetting its parser state will lose the body.

This breaks many interop scenarios between Spring Boot and Python ASGI apps.

I'm happy to submit a PR if maintainers are open to it. Thanks for your time and for maintaining this great project!

贡献指南

这个仓库没有索引到贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

调研方向

解析器回调位于 parser.pyx;首先跟踪 cb_on_headers_complete 和现有的 upgrade 处理,然后检查 Python 解析器包装器和可复现的解析器测试。重现分块的 h2c 请求,并验证在忽略 upgrade 后 body 仍得到保留,同时确保相关解析器测试通过。

由索引模型根据 Issue 内容生成。

评估

技术栈
python
领域
backend, networking
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
基本清楚
新手友好度
42/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。