refly-ai / refly-ai/refly

Server-Side Request Forgery in misc/scrape weblink endpoint

Open
#2,280 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
7.5k
Forks
729
PR merge metrics
No merged PRs in 30d

Description

The endpoint POST /v1/misc/scrape (apps/api/src/modules/misc/misc.controller.ts) takes a url field from the request body and passes it directly to the shared scrapeWeblink utility (packages/utils/src/scrape-weblink.ts), which calls fetch() on it with no validation of the scheme, host, or resolved IP address and follows redirects. The fetched page's title and meta description are then returned in the response.

Any registered user (email signup is enabled by default) can therefore make the backend issue HTTP requests to internal-only addresses such as cloud metadata services, internal dashboards, or other services on the deployment network, and read back the page title and description. I confirmed the behavior by running the exact scrapeWeblink logic against a local internal-only HTTP server: the request reached it and the server's internal title and meta description were returned to the caller. The same unguarded fetch() is also reachable through dumpFileFromURL when re-hosting markdown image URLs, which stores the full response body.

Suggested fix: Before fetching, resolve the target hostname and reject requests whose resolved IP falls in private, loopback, link-local, or cloud-metadata ranges (including IPv4-mapped IPv6 and 0.0.0.0); restrict the scheme to http/https; and re-validate the destination after each redirect (or disable redirects) to prevent DNS-rebinding and redirect-based bypasses. Apply the same guard centrally so both the scrape endpoint and dumpFileFromURL are covered.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with apps/api/src/modules/misc/misc.controller.ts and packages/utils/src/scrape-weblink.ts, then trace the shared fetch path used by scrapeWeblink and dumpFileFromURL. Verify behavior against the reported internal-only HTTP server scenario. Done means both paths reject unsafe schemes and resolved private, loopback, link-local, metadata, IPv4-mapped, and 0.0.0.0 destinations, including redirect or DNS-rebinding bypasses.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
api, backend, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.