PipedreamHQ / PipedreamHQ/pipedream

[APP] Hydrafetch

Open
#21,734 0 comments 0 reactions 0 assignees View on GitHub
app enhancement
Dominant language
JavaScript
Stars
11.7k
Forks
5.8k
Avg merge
3d 10h
Merged PRs (30d)
102

Description

**Name of app / service**

Hydrafetch — https://hydrafetch.com

We build Hydrafetch and would like to get it into the registry. It is a web data API: send a URL, get back clean markdown or structured data with the navigation, banners and boilerplate stripped out. Scrape, map, crawl, search, extract and brand resolution sit behind one API with a single response shape.

**Link to developer documentation**

- Docs: https://docs.hydrafetch.com
- OpenAPI spec: https://api.hydrafetch.com/openapi.json
- Auth is a single `X-API-Key` header. Signup at https://app.hydrafetch.com is self-serve and gives 500 credits on a work email with no card, so components can be built and tested without contacting us first.

**Is lack of support preventing you from moving forward, or do you have a workaround?**

There is a workaround. The API is plain REST with one header, so it works today through the HTTP Request action. An app entry would remove the hand-built requests and the key handling, the same way the existing `firecrawl`, `tavily`, `exa` and `scrapingbee` apps do.

**Are there specific actions or triggers you'd like to see for this app? Please let us know here or use the Action and Trigger issue templates to open requests for each!**

Actions: scrape a URL to markdown, HTML, links or structured data; map a site's URLs without fetching bodies; crawl a whole site; get crawl status and results; batch scrape an explicit list of URLs; search the web with results optionally scraped; extract schema-shaped JSON from one or many URLs; resolve a company brand from a domain; capture a screenshot or harvest page images.

Sources (webhook-based): crawl job completed, and batch job completed. Both endpoints accept a `webhook` parameter and expose delivery history at `/v1/web/crawl/{id}/deliveries` and `/v1/web/batch/{id}/deliveries`. Webhooks need a paid plan, so we are happy to provide a workspace to whoever builds the sources.

Happy to write the components ourselves once the app directory exists, following the Component Guidelines and mirroring the shape of `components/firecrawl`.

Contributor guide

Open the contributing guide

Research direction

Start with the Component Guidelines and components/firecrawl, then review Hydrafetch’s developer documentation and OpenAPI spec to map its actions and webhook sources. Done means a Hydrafetch app entry supports the requested API capabilities and X-API-Key authentication while matching existing app conventions.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, openapi
Domain
api, backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.