Joystream / Joystream/joystream

[DWG] Consider using Cloudflare proxy

Open
#5,048 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Rust
Stars
1.4k
Forks
116
PR merge metrics
No merged PRs in 30d

Description

One thing I have experimented with is using Cloudflare proxy for my distribution node. In the past I have used `gateway.joyutils.org` which was a Cloudflare Worker function that would just proxy the request to my node. Using that has shown big bandwidth gains, but at the same time has increased latency quite a bit. You can find those results in the CDN performance report - https://joystream.notion.site/Joystream-CDN-performance-part-1-c8aff72f95ca4e86866079f81784aacb?pvs=4

Recently I've tried a simpler approach - just enabling Cloudflare proxy at the DNS level. Whenever you add a DNS record through Cloudflare, you have an option of enabling their proxy for that record. As I understand it, when enabled, all the requests are not routed directly to the backend server but rather to the closest Cloudflare data centre and then they use Cloudflare private network for the rest of their travel. This is how the enabled proxy looks like in DNS settings:
![CleanShot 2024-01-18 at 10 29 29@2x](https://github.com/Joystream/joystream/assets/12646744/8895ebe4-fbf6-4561-9274-5dabdd527e4f)
You can read more about it in Cloudflare docs: https://developers.cloudflare.com/dns/manage-dns-records/reference/proxied-dns-records/

Here you can see the difference in performance from our synthethic test. Without proxy:
![CleanShot 2024-01-17 at 10 16 25@2x](https://github.com/Joystream/joystream/assets/12646744/4531bb9a-6b9b-42de-92ea-39acd73448ef)

With proxy:
![CleanShot 2024-01-17 at 10 16 47@2x](https://github.com/Joystream/joystream/assets/12646744/cbcc4ade-4d87-4d0e-b9b0-85f03f8b01cf)

As you can see, the difference in latency is pretty massive. I think we should consider enabling this kind of proxy for some of our distribution nodes.

Things to keep in mind:
1. We cannot be over-reliant on Cloudflare - disruption in their services shouldn't be critical to our CDN.
2. Requests proxied through Cloudflare would most likely come from their IP addresses which can disrupt our dashboards that gives us an idea of where our users are. We should look into whether we can somehow extract original user IP from the requests.
3. Cloudflare can block some users, for example those using Tor network.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.