langgenius / langgenius/dify

Workflow remote file metadata falls back to URL guesses when HEAD is unsupported

Open
#42,313 2 comments 1 reaction 0 assignees View on GitHub
Dominant language
TypeScript
Stars
156k
Forks
24.6k
Avg merge
22h 9m
Merged PRs (30d)
610

Description

### Self Checks

- [x] I have read the Contributing Guide and Language Policy.
- [x] I have searched for existing issues and pull requests, including closed ones.
- [x] I confirm that I am using English to submit this report.

### Dify version

main

### Cloud or Self Hosted

Both

### Problem

Workflow remote-file normalization obtains metadata in `get_remote_file_info()` through a `HEAD` request only. When an origin returns `405 Method Not Allowed` or `501 Not Implemented` for `HEAD`, while a normal `GET` succeeds, the helper skips response metadata and falls back to guesses from the URL path.

For signed download endpoints or URLs without a filename, this means a file with valid `Content-Disposition`, `Content-Type`, and `Content-Length` on `GET` is represented as a random `.bin` file with an unknown size. That can make workflow file-type normalization lose the actual file type even though the source is otherwise retrievable.

This is a separate path from the web reader fallback addressed by #42242. The affected workflow path is:

`factories/file_factory/builders.py` -> `factories/file_factory/remote.py:get_remote_file_info()`.

### Minimal reproduction

Use a remote URL such as `https://example.test/download` with these responses:

- `HEAD /download` -> `405 Method Not Allowed`
- `GET /download` -> `200 OK`, with:

```text
Content-Disposition: attachment; filename="report.pdf"
Content-Type: application/pdf
Content-Length: 123
```

Current `get_remote_file_info()` returns a URL-derived or generated filename (for example `.bin`), an empty/guessed MIME type, and `-1` size because it only reads metadata after a `200` HEAD response.

Expected behavior: for `405` and `501` only, fall back to a streaming `GET`, use its headers to derive metadata, and close the response without buffering the file body. Existing behavior for successful HEAD responses and other HTTP statuses should remain unchanged.

### Proposed scope

- In `api/factories/file_factory/remote.py`, handle HEAD `405`/`501` by requesting a streaming GET and extracting only response headers.
- Add focused tests in `api/tests/unit_tests/factories/test_file_factory.py` for both statuses and a URL without a filename.
- Keep SSRF-safe access through `remote_fetcher` and do not alter unrelated remote-file download behavior.

Contributor guide

Open the contributing guide

Research direction

Start with api/factories/file_factory/remote.py and get_remote_file_info(), then read the related cases in api/tests/unit_tests/factories/test_file_factory.py. Run the focused file-factory tests and add coverage for HEAD responses returning 405 and 501, including a URL without a filename. Done means streaming GET headers provide the metadata, the response is closed without buffering the body, successful HEAD behavior remains unchanged, and the tests pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
85/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.