[import] CDN protected sites cannot be proxied
- Dominant language
- JavaScript
- Stars
- 68
- Forks
- 72
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 8
Description
Run `hlx import` and try to import page https://www.globe.com.ph/business/enterprise.html. Nothing happens, the proxy returns 403.
Try to open proxy page in another browser tab: http://localhost:3001/business/enterprise.html?host=https%3A%2F%2Fwww.globe.com.ph and you get this:

The proxy request has the exact same headers than the browser request. Cloudflare puts a lot of effort from proxy / scripts / bots (well, no humans...) to not access the site. I do not think we can workaround this.
Contributor guide
Research direction
Start by running `hlx import` against the Globe enterprise URL and open the supplied localhost proxy URL in another browser tab. Compare the proxy and browser requests, then determine whether a workable approach exists for the Cloudflare 403; done would require a reproducible fix or a documented decision that protected sites cannot be imported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- cli, networking
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100