crewAIInc / crewAIInc/crewAI

[BUG] DOCXLoader downloads remote files without a size limit

Open
#7,631 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
58.8k
Forks
8.5k
Avg merge
1d 15h
Merged PRs (30d)
109

Description

Description

DOCXLoader downloads remote documents with safe_get(...).content, so the entire response is buffered without a size limit and then copied to a temporary file. An oversized remote response can therefore consume unbounded memory and temporary-disk space before DOCX parsing begins.

The adjacent PDFLoader already uses the shared safe_get_bounded helper with a 50 MiB default and a per-call max_bytes override. DOCX URL loading should apply the same resource boundary.

This report was prepared with AI assistance. I cannot apply repository labels as an external contributor; please add the required llm-generated label.

Steps to reproduce

  1. Call DOCXLoader.load(SourceContent("https://example.com/large.docx")) against a server returning a very large body.
  2. Observe that _download_from_url() reads response.content without a ceiling.
  3. The full response is retained in memory and then written to a temporary file before parsing.

Expected behavior

Remote DOCX downloads should stop after a bounded number of decoded bytes, close the response, and avoid creating a temporary file for an oversized body. Callers should be able to lower or raise the default through max_bytes, matching PDFLoader.

Screenshots/Code snippets

Current download path:

response = safe_get(url, headers=headers, timeout=30)
response.raise_for_status()
temp_file.write(response.content)

Operating System

macOS Sonoma

Python Version

3.13 (supported by the repository; not listed in the issue template)

crewAI Version

Current main (3831e8b6c86f)

crewAI Tools Version

Current main (3831e8b6c86f)

Virtual Environment

Venv (uv)

Evidence

The existing path accepts a body larger than 50 MiB because no byte limit is supplied. Focused offline tests confirm that the shared bounded helper rejects oversized responses before temporary-file creation and closes streamed responses on all paths.

Possible Solution

Use safe_get_bounded with a 50 MiB default, preserve custom headers, and forward an optional max_bytes override. Keep the existing parse-time finally cleanup for accepted temporary files.

Additional context

The limit covers decoded download bytes, not ZIP decompression expansion inside a DOCX archive. Archive-expansion hardening is a separate concern. No matching open issue or pull request was found before filing.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at DOCXLoader._download_from_url() and compare its safe_get path with the adjacent PDFLoader use of safe_get_bounded. Verify the focused offline tests for oversized responses, response closure, temporary-file creation, and the max_bytes override; done means remote DOCX downloads enforce the default and caller-supplied byte limits without regressing accepted-file cleanup.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.