pydantic / pydantic/httpx2

Support async file types in `files = {}` and `content = ...`

Open
#597 26 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.5k
Forks
78
Avg merge
8h 59m
Merged PRs (30d)
24

Description

Originally opened by @tomchristie on 2021-04-30 09:40:18 in encode/httpx

We ought to support the following cases.

Raw upload content from an async file interface:

import httpx
import trio

async def main():
    async with httpx.AsyncClient() as client:
        async with await trio.open_file(...) as f:
            client.post("https://www.example.com", content=f)

trio.run(main)

Multipart file upload from an async file interface:

import httpx
import trio

async def main():
    async with httpx.AsyncClient() as client:
        async with await trio.open_file(...) as f:
            client.post("https://www.example.com", file={"upload": f})

trio.run(main)

We probably want to ensure that we're supporting both trio, anyio (Which have the same interfaces), and perhaps also `aiofiles. So eg, also supporting the following...

# Supporting the same as above but using `asyncio`, with `anyio` for the file operations.
import anyio
import asyncio
import httpx

async def main():
    async with httpx.AsyncClient() as client:
        async with await anyio.open_file(...) as f:
            client.post("https://www.example.com", content=f)

asyncio.run(main())

The content=... case is a little simpler than the data=... case, since it really just need an async variant of peek_filelike_length, and a minor update to the ._content.encode_content() function.

Also fiddly is what the type annotations ought to look like.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the async content and multipart upload paths, especially peek_filelike_length and ._content.encode_content(), then trace how content=, data=, and files={} are handled. Check the existing annotations and tests around these entry points; done means the requested async file interfaces work for raw content and multipart uploads without regressing current cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.