Add a pluggable media uploader so large video downloads don't OOM serverless runtimes
@tombeckenham is already working on this.
Since Sep 17, 2026.
- Dominant language
- TypeScript
- Stars
- 3.1k
- Forks
- 331
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 160
Description
Spinning out a follow-up flagged in PR #409 review (https://github.com/TanStack/ai/pull/409#discussion_r2218600385 / r2218663501).
OpenAICompatibleVideoAdapter.getVideoUrl falls through to videos.downloadContent/content/getContent/download and a raw fetch for backends that don't return a CDN URL on retrieve(). In every fallback path we read the entire video into an ArrayBuffer, base64-encode it, and emit a data: URL. PR #409 added a >10 MiB warning (warnIfLargeMediaBuffer) but the underlying behaviour is still unsafe on Workers / Lambda / small isolates.
This will almost certainly blow up workers. It's pretty common for videos to be bigger than 100mb. — @tombeckenham
Downloading the video and base64'ing it will cause most backends to run out of memory. I'm wondering if we support these older methods at all. At the very least we should add a warning with logger and look at whether buffers are over e.g. 10mb. It's quite hard to figure out where memory issues come from as they don't necessarily occur at the point this blob would be created. I'm wondering if we might want to add a media uploader function to the the constructor in a future PR — @tombeckenham
Scope
- Design a
mediaUploader(orMediaUploadStrategy) constructor option onOpenAICompatibleVideoAdapter(and likely image/audio adapters too) that takes a stream-or-blob and returns a public URL. - Default behaviour: if no uploader is configured AND the SDK doesn't expose a
retrieve()-time CDN URL, return aVideoUrlResultthat surfaces "upstream URL not available; provide amediaUploaderto download/host the bytes" rather than silently base64ing. - Promote the current
warnIfLargeMediaBufferto flow through theInternalLoggerinstead ofconsole.warnonce a logger is plumbed through these methods. - Apply the same strategy to TTS / image adapters where a base64 data URL is emitted today (see also the response-input-file base64 follow-up @tombeckenham mentioned).
Related
- PR #409 review thread on
video.ts:148andvideo.ts:165. - The existing
responses-text.tsbase64 data-URI thread (https://github.com/TanStack/ai/pull/409#discussion_r2218493238) — Tom mentioned an in-flight PR removing that for input attachments; this issue covers the symmetrical concern on the output side.
cc @tombeckenham
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.