Title Gaussian Splat memory accounting does not fit existing cacheBytes model
- Dominant language
- JavaScript
- Stars
- 15.7k
- Forks
- 3.9k
- Avg merge
- 4d 6h
- Merged PRs (30d)
- 34
Description
Gaussian Splats currently do not fit well into the existing cacheBytes memory model used by traditional 3D Tiles.
The current cache accounting does not clearly distinguish between CPU memory, GPU memory, transient upload memory, and persistent runtime memory. This is especially noticeable for Gaussian Splats because their memory behavior is different from regular mesh tiles.
For traditional 3D Tiles, geometry data is usually uploaded to the GPU and the CPU-side buffers can often be released afterward. For Gaussian Splats, some CPU-side data needs to remain resident during runtime, while GPU textures are also allocated at the same time.
After https://github.com/CesiumGS/cesium/issues/13483 started reporting GS memory through `geometryByteLength`, moderately large Gaussian Splat tilesets can quickly exceed the default `cacheBytes` budget and trigger `memoryAdjustedScreenSpaceError` behavior unless users manually increase the cache limit.
This raises a broader question about what `cacheBytes` and `geometryByteLength` should represent for Gaussian Splats. Are they intended to account for GPU memory, CPU memory, transient upload memory, or the total effective runtime footprint?
It may be worth investigating whether Gaussian Splats need a different memory accounting strategy, such as clearer CPU/GPU memory separation, GS-specific cache behavior, or reducing duplicated GS buffers in memory. It may also be worth clarifying which CPU-side attributes truly need to remain resident. For example, depth sorting may only require positions.
Contributor guide
Research direction
The issue names no implementation files or tests. Begin by tracing the existing cacheBytes and geometryByteLength accounting for Gaussian Splat tiles, comparing CPU buffers, GPU textures, transient upload memory, and persistent runtime data. Done means the intended memory footprint is defined and an appropriate accounting strategy is identified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- computer-graphics, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100