Compress caching information
- Dominant language
- Python
- Stars
- 9
- Forks
- 31
- Avg merge
- 3d 19h
- Merged PRs (30d)
- 16
Description
In order to better scale for multiple requests. We can compress the cache information using a fast compression strategy (such as gzip or zstd).
A few examples of compressed cache information savings using gzip over a json serialized object.
```
=================== treeCommits-4434adf90c8187f7d535a32598f74282e3f3903316089c907a408abf96681cbe
--- Deep Compression Stats ---
Raw Deep Object Size (RAM): 11,644 bytes
Compressed Size (RAM): 1,410 bytes
Compression Ratio: 8.26x smaller
RAM Space Saved: 87.9%
------------------------------
[17/Jul/2026 13:15:54] "GET /api/tree/stable/linux-6.12.y/commits?origin=maestro&git_url=https:%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Fstable%2Flinux.git HTTP/1.1" 200 1494
[17/Jul/2026 13:15:55] "GET /api/build/maestro:6a490326bb83f6a3e88f3c18 HTTP/1.1" 200 953
=================== treeDetailsBuildsData-0d86f4f1fe69154adfd1b77efbbcdd84679f9dac38ae1d564d72d473c3a8907d
--- Deep Compression Stats ---
Raw Deep Object Size (RAM): 119,399 bytes
Compressed Size (RAM): 3,830 bytes
Compression Ratio: 31.17x smaller
RAM Space Saved: 96.8%
------------------------------
=================== treeCommitHistoryHashesAggregatedNoCompatibles-d78005bcecc540871658eabd7dec3143e007e360a47ce6817fdb89944efa17f9
--- Deep Compression Stats ---
Raw Deep Object Size (RAM): 277,642 bytes
Compressed Size (RAM): 2,140 bytes
Compression Ratio: 129.74x smaller
RAM Space Saved: 99.2%
------------------------------
[17/Jul/2026 13:15:55] "GET /api/tree/commits-history?origin=maestro&git_url=https:%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Fstable%2Flinux.git&git_branch=linux-6.12.y&types=builds&tree_name=stable&commit_hashes=296aabce459470a4c1b68ffd0c0c0920e563aaad,1d3a00d3bacff25652c96e1527610c69e91f7c38,26deb9f9c427c0382f855546d23dba76c49e680c,c4ffbe29c40ed851601bce640d5ead48eaaae08d,2538fbeff8a94ee2b54eb09d92209e24a1e650d4,51eba7e6c91ada319dc9ebee3f88e4472b5efbc4 HTTP/1.1" 200 525
=================== treeDetailsBuilds-0d86f4f1fe69154adfd1b77efbbcdd84679f9dac38ae1d564d72d473c3a8907d
--- Deep Compression Stats ---
Raw Deep Object Size (RAM): 94,511 bytes
Compressed Size (RAM): 3,689 bytes
Compression Ratio: 25.62x smaller
RAM Space Saved: 96.1%
------------------------------
[17/Jul/2026 13:15:55] "GET /api/tree/stable/linux-6.12.y/296aabce459470a4c1b68ffd0c0c0920e563aaad/builds?origin=maestro HTTP/1.1" 200 3698
=================== treeDetailsRollup-0d86f4f1fe69154adfd1b77efbbcdd84679f9dac38ae1d564d72d473c3a8907d
--- Deep Compression Stats ---
Raw Deep Object Size (RAM): 997,633 bytes
Compressed Size (RAM): 10,551 bytes
Compression Ratio: 94.55x smaller
RAM Space Saved: 98.9%
------------------------------
[17/Jul/2026 13:15:55] "GET /api/tree/stable/linux-6.12.y/296aabce459470a4c1b68ffd0c0c0920e563aaad/summary?origin=maestro HTTP/1.1" 200 3358
=================== treeCommitHistoryHashesAggregatedNoCompatibles-f08021d43bda85ae05c447dbd9c2ba94131e8d27a6c1b71fdad9305c945a1900
--- Deep Compression Stats ---
Raw Deep Object Size (RAM): 352,767 bytes
Compressed Size (RAM): 2,985 bytes
Compression Ratio: 118.18x smaller
RAM Space Saved: 99.2%
------------------------------
[17/Jul/2026 13:16:03] "GET /api/tree/commits-history?origin=maestro&git_url=https:%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Fstable%2Flinux.git&git_branch=linux-6.12.y&types=tests&tree_name=stable&commit_hashes=296aabce459470a4c1b68ffd0c0c0920e563aaad,1d3a00d3bacff25652c96e1527610c69e91f7c38,26deb9f9c427c0382f855546d23dba76c49e680c,c4ffbe29c40ed851601bce640d5ead48eaaae08d,2538fbeff8a94ee2b54eb09d92209e24a1e650d4,51eba7e6c91ada319dc9ebee3f88e4472b5efbc4 HTTP/1.1" 200 631
=================== treeDetailsBoots-0d86f4f1fe69154adfd1b77efbbcdd84679f9dac38ae1d564d72d473c3a8907d
--- Deep Compression Stats ---
Raw Deep Object Size (RAM): 347,704 bytes
Compressed Size (RAM): 10,622 bytes
Compression Ratio: 32.73x smaller
RAM Space Saved: 96.9%
------------------------------
[17/Jul/2026 13:16:29] "GET /api/tree/stable/linux-6.12.y/296aabce459470a4c1b68ffd0c0c0920e563aaad/boots?origin=maestro HTTP/1.1" 200 4330
=================== treeCommitHistoryHashesAggregatedNoCompatibles-5ace2e79803e5d006b0402d9cb2de7b547c2147b17c1b5d15acabfa67967cb66
--- Deep Compression Stats ---
Raw Deep Object Size (RAM): 159,218 bytes
Compressed Size (RAM): 1,540 bytes
Compression Ratio: 103.39x smaller
RAM Space Saved: 99.0%
------------------------------
[17/Jul/2026 13:16:29] "GET /api/tree/commits-history?origin=maestro&git_url=https:%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Fstable%2Flinux.git&git_branch=linux-6.12.y&types=boots&tree_name=stable&commit_hashes=296aabce459470a4c1b68ffd0c0c0920e563aaad,1d3a00d3bacff25652c96e1527610c69e91f7c38,26deb9f9c427c0382f855546d23dba76c49e680c,c4ffbe29c40ed851601bce640d5ead48eaaae08d,2538fbeff8a94ee2b54eb09d92209e24a1e650d4,51eba7e6c91ada319dc9ebee3f88e4472b5efbc4 HTTP/1.1" 200 612
[17/Jul/2026 13:16:35] "GET /api/test/maestro:6a4920abbb83f6a3e88ff34e HTTP/1.1" 200 709
[17/Jul/2026 13:16:36] "GET /api/proxy/?url=https%3A%2F%2Ffiles.kernelci.org%2Flogexcerpt%2F8349496689ffd010f5e5a0918fc7dcd178f17df64754576a2a01054c4b012488%2Flogexcerpt.txt.gz HTTP/1.1" 200 5180
=================== treeDetailsTests-0d86f4f1fe69154adfd1b77efbbcdd84679f9dac38ae1d564d72d473c3a8907d
--- Deep Compression Stats ---
Raw Deep Object Size (RAM): 50,329,625 bytes
Compressed Size (RAM): 460,648 bytes
Compression Ratio: 109.26x smaller
RAM Space Saved: 99.1%
------------------------------
[17/Jul/2026 13:16:38] "GET /api/tree/stable/linux-6.12.y/296aabce459470a4c1b68ffd0c0c0920e563aaad/tests?origin=maestro HTTP/1.1" 200 247813
[17/Jul/2026 13:16:49] "GET /api/hardware/?startTimestampInSeconds=1783863000&endTimestampInSeconds=1784295000&origin=maestro HTTP/1.1" 200 3100
=================== hardwareSelectors-b4b8676dd4a99244f976027019cb2ac0be2e2f0dd6e66c6411ac8ca7c75c746c
--- Deep Compression Stats ---
Raw Deep Object Size (RAM): 510,098 bytes
Compressed Size (RAM): 34,406 bytes
Compression Ratio: 14.83x smaller
RAM Space Saved: 93.3%
------------------------------
[17/Jul/2026 13:16:50] "GET /api/hardware/selectors/?origin=maestro HTTP/1.1" 200 33767
[17/Jul/2026 13:16:53] "OPTIONS /api/hardware/acer-cbv514-1h-34uz-brya/summary HTTP/1.1" 200 0
=================== hardwareTreesHeadCommits-35a0bf9fb58aab359210013bdbd27ce40f1b595608ff3f6b3a5e0da6272bc451
--- Deep Compression Stats ---
Raw Deep Object Size (RAM): 804 bytes
Compressed Size (RAM): 148 bytes
Compression Ratio: 5.43x smaller
RAM Space Saved: 81.6%
------------------------------
=================== hardwareDetailsSummary-4b8fceb29a4c5129ff6c2bd0a33977584330d920d76f10726349d5ca203c1e33
--- Deep Compression Stats ---
Raw Deep Object Size (RAM): 37,328 bytes
Compressed Size (RAM): 855 bytes
Compression Ratio: 43.66x smaller
RAM Space Saved: 97.7%
------------------------------
[17/Jul/2026 13:17:05] "POST /api/hardware/acer-cbv514-1h-34uz-brya/summary HTTP/1.1" 200 1042
[17/Jul/2026 13:17:05] "OPTIONS /api/hardware/acer-cbv514-1h-34uz-brya/commit-history HTTP/1.1" 200 0
[17/Jul/2026 13:17:05] "OPTIONS /api/hardware/acer-cbv514-1h-34uz-brya HTTP/1.1" 200 0
[17/Jul/2026 13:17:08] "POST /api/hardware/acer-cbv514-1h-34uz-brya/commit-history HTTP/1.1" 200 1228
=================== hardwareDetailsTreeData-962170cbbe0ec3808e99b9565fc8d16e026f9b06a64b4ce6844d217dffbc3210
--- Deep Compression Stats ---
Raw Deep Object Size (RAM): 7,003 bytes
Compressed Size (RAM): 426 bytes
Compression Ratio: 16.44x smaller
RAM Space Saved: 93.9%
------------------------------
=================== hardwareDetailsFullData-8951206d9caf3f6e2fd23fadc21d7804d8c3ccf480622f6ea79a89046642cdb2
--- Deep Compression Stats ---
Raw Deep Object Size (RAM): 5,369,090 bytes
Compressed Size (RAM): 58,929 bytes
Compression Ratio: 91.11x smaller
RAM Space Saved: 98.9%
------------------------------
[17/Jul/2026 13:17:14] "POST /api/hardware/acer-cbv514-1h-34uz-brya HTTP/1.1" 200 27233
[17/Jul/2026 13:17:17] "GET /api/build/maestro:6a5977c53427a5b0fbeaa8be HTTP/1.1" 200 927
```
Contributor guide
Research direction
The issue names no files, tests, or entry points. Start by locating the cache serialization and storage paths, then assess where compression can be applied and whether gzip or zstd fits the existing dependencies. Done should preserve cache reads and writes while reducing cache memory use for the demonstrated data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100