CesiumGS / CesiumGS/cesium

Title Gaussian Splat memory accounting does not fit existing cacheBytes model

Open
#13,503 1 comment 0 reactions 0 assignees View on GitHub
category - gaussian splatting type - enhancement
Dominant language
JavaScript
Stars
15.7k
Forks
3.9k
Avg merge
4d 6h
Merged PRs (30d)
34

Description

Gaussian Splats currently do not fit well into the existing cacheBytes memory model used by traditional 3D Tiles.

The current cache accounting does not clearly distinguish between CPU memory, GPU memory, transient upload memory, and persistent runtime memory. This is especially noticeable for Gaussian Splats because their memory behavior is different from regular mesh tiles.

For traditional 3D Tiles, geometry data is usually uploaded to the GPU and the CPU-side buffers can often be released afterward. For Gaussian Splats, some CPU-side data needs to remain resident during runtime, while GPU textures are also allocated at the same time.

After https://github.com/CesiumGS/cesium/issues/13483 started reporting GS memory through `geometryByteLength`, moderately large Gaussian Splat tilesets can quickly exceed the default `cacheBytes` budget and trigger `memoryAdjustedScreenSpaceError` behavior unless users manually increase the cache limit.

This raises a broader question about what `cacheBytes` and `geometryByteLength` should represent for Gaussian Splats. Are they intended to account for GPU memory, CPU memory, transient upload memory, or the total effective runtime footprint?

It may be worth investigating whether Gaussian Splats need a different memory accounting strategy, such as clearer CPU/GPU memory separation, GS-specific cache behavior, or reducing duplicated GS buffers in memory. It may also be worth clarifying which CPU-side attributes truly need to remain resident. For example, depth sorting may only require positions.

Contributor guide

Open the contributing guide

Research direction

The issue names no implementation files or tests. Begin by tracing the existing cacheBytes and geometryByteLength accounting for Gaussian Splat tiles, comparing CPU buffers, GPU textures, transient upload memory, and persistent runtime data. Done means the intended memory footprint is defined and an appropriate accounting strategy is identified.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
computer-graphics, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.