internetarchive / internetarchive/openlibrary
kTLS with Docker and Nginx
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 2k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 138
Description
I was wondering if it might be worth enabling [kTLS](https://www.kernel.org/doc/html/latest/networking/tls-offload.html) on any Nginx servers that serve a lot of static content or cached responses.
I was looking at how Netflix can [serve 800Gb/s of TLS encrypted video content from a single server](https://people.freebsd.org/~gallatin/talks/euro2022.pdf), and a large part of it appears to be kTLS.
I also saw a blog post from Nginx about improving [Nginx performance with Kernel TLS and SSL_sendfile()](https://www.nginx.com/blog/improving-nginx-performance-with-kernel-tls/) using recent versions of Linux and FreeBSD, so I gave it a shot in Linux and Docker.
I did a test similar to the (curiously broken) bash loop in the blog post, and with kTLS enabled my CPU usage went from around 48-50% to around 27-29% while serving a 1 GB image 100 times, and throughput about doubled.
As it was just a test I didn't put a huge amount of effort into getting things working properly, so although I tried to load the kernel module from within `docker-compose.yml` using `cap_admin`, I ended up just loading it on the host with `sudo modprobe tls`.
I've attached the files I used, including some a dummy SSL cert.
[nginx_ktls.zip](https://github.com/internetarchive/openlibrary/files/9935640/nginx_ktls.zip)
Just raising the idea for evaluation.
Contributor guide
Research direction
Start by reviewing the attached nginx_ktls.zip and its docker-compose.yml, then read the linked Nginx and kernel TLS references. Check whether the proposed setup fits Open Library's deployment and static or cached response serving; done means a maintainer-approved evaluation or a clearly scoped implementation path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, linux, nginx
- Domain
- devops, infrastructure, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100