PipedreamHQ / PipedreamHQ/pipedream
[APP] Hydrafetch
- Dominant language
- JavaScript
- Stars
- 11.7k
- Forks
- 5.8k
- Avg merge
- 3d 10h
- Merged PRs (30d)
- 102
Description
**Name of app / service**
Hydrafetch — https://hydrafetch.com
We build Hydrafetch and would like to get it into the registry. It is a web data API: send a URL, get back clean markdown or structured data with the navigation, banners and boilerplate stripped out. Scrape, map, crawl, search, extract and brand resolution sit behind one API with a single response shape.
**Link to developer documentation**
- Docs: https://docs.hydrafetch.com
- OpenAPI spec: https://api.hydrafetch.com/openapi.json
- Auth is a single `X-API-Key` header. Signup at https://app.hydrafetch.com is self-serve and gives 500 credits on a work email with no card, so components can be built and tested without contacting us first.
**Is lack of support preventing you from moving forward, or do you have a workaround?**
There is a workaround. The API is plain REST with one header, so it works today through the HTTP Request action. An app entry would remove the hand-built requests and the key handling, the same way the existing `firecrawl`, `tavily`, `exa` and `scrapingbee` apps do.
**Are there specific actions or triggers you'd like to see for this app? Please let us know here or use the Action and Trigger issue templates to open requests for each!**
Actions: scrape a URL to markdown, HTML, links or structured data; map a site's URLs without fetching bodies; crawl a whole site; get crawl status and results; batch scrape an explicit list of URLs; search the web with results optionally scraped; extract schema-shaped JSON from one or many URLs; resolve a company brand from a domain; capture a screenshot or harvest page images.
Sources (webhook-based): crawl job completed, and batch job completed. Both endpoints accept a `webhook` parameter and expose delivery history at `/v1/web/crawl/{id}/deliveries` and `/v1/web/batch/{id}/deliveries`. Webhooks need a paid plan, so we are happy to provide a workspace to whoever builds the sources.
Happy to write the components ourselves once the app directory exists, following the Component Guidelines and mirroring the shape of `components/firecrawl`.
Contributor guide
Research direction
Start with the Component Guidelines and components/firecrawl, then review Hydrafetch’s developer documentation and OpenAPI spec to map its actions and webhook sources. Done means a Hydrafetch app entry supports the requested API capabilities and X-API-Key authentication while matching existing app conventions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, openapi
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100