MagicStack / MagicStack/httptools

[Bug] Request body lost when Upgrade: h2c + Transfer-Encoding: chunked is used

Abierto
#124 0 comentarios 2 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Lenguaje dominante
Python
Estrellas
1.3k
Forks
107
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

Overview

When sending a POST request from a Java RestClient (Spring Boot 3.2+, Java 21) to a FastAPI backend running on Uvicorn + httptools, we encountered a strange issue where the request body was missing.

The request looked like this:

POST /endpoint HTTP/1.1
Host: my-api.com
Upgrade: h2c
Connection: Upgrade, HTTP2-Settings
Transfer-Encoding: chunked
Content-Type: application/json

3\r\nabc\r\n0\r\n\r\n

On the server side, Uvicorn logs showed:

  • Unsupported upgrade request
  • No request body
  • Invalid HTTP request received

But when we routed the same request through ngrok or used RestTemplate instead of RestClient, it worked fine.


🔍 Root Cause

After analyzing Uvicorn’s httptools_impl.py and httptools parser behavior, we found this:

  • Upgrade: h2c is ignored by Uvicorn (as expected).
  • But internally, httptools still enters the upgrade state.
  • Since the upgrade is ignored and the parser is not reset, no body is parsed.
  • This violates RFC 7230 §6.7, which allows the server to ignore upgrades and proceed normally.

Proposed Fix

Patch parser.pyx to resume HTTP/1.1 parsing after upgrade is ignored:

cdef int cb_on_headers_complete(cparser.llhttp_t* parser) except -1:
    cdef HttpParser pyparser = <HttpParser>parser.data
    try:
        if parser.upgrade and not pyparser._should_upgrade():
            cparser.llhttp_resume_after_upgrade(parser)
        pyparser._on_headers_complete()
    except BaseException as ex:
        pyparser._last_error = ex
        return -1
    return 0

Also expose this from Python:

def resume_after_upgrade(self):
    httptools.llhttp_resume_after_upgrade(self.cparser)

Then frameworks like Uvicorn can call it in:

def on_headers_complete(self):
    if self.upgrade and self.upgrade.lower() != b"websocket":
        self.parser.resume_after_upgrade()

Reproducible Test

def test_chunked_body_with_ignored_upgrade():
    headers = {
        "Upgrade": "h2c",
        "Connection": "Upgrade",
        "Transfer-Encoding": "chunked"
    }
    body = b"4\r\ntest\r\n0\r\n\r\n"
    request = b"POST / HTTP/1.1\r\n" + headers_to_bytes(headers) + b"\r\n" + body

    parser = HttpRequestParser(TestProtocol())
    parser.feed_data(request)

    assert protocol.body == b"test"

Why it matters

This is RFC-compliant behavior that should be supported.

RestClient in Java 21+ sends Upgrade: h2c by default.

Any server not resetting its parser state will lose the body.

This breaks many interop scenarios between Spring Boot and Python ASGI apps.

I'm happy to submit a PR if maintainers are open to it. Thanks for your time and for maintaining this great project!

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Línea de trabajo

El callback del parser está en parser.pyx; empieza siguiendo cb_on_headers_complete y el manejo existente de upgrade, y después inspecciona el wrapper del parser de Python y la prueba reproducible del parser. Reproduce la solicitud h2c fragmentada y verifica que el body se conserve después de un upgrade ignorado, con las pruebas relevantes del parser superadas.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
python
Área
backend, networking
Tipo de issue
Error
Dificultad
4/5
Tiempo estimado
3-5 días
Estado de actividad
Estancado
Claridad
Bastante claro
Aptitud para principiantes
42/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.