CesiumGS / CesiumGS/cesium

Cannot combine splat- and non-splat content in one tileset

Open
#13,312 0 comments 0 reactions 0 assignees View on GitHub
category - gaussian splatting type - enhancement
Dominant language
JavaScript
Stars
15.8k
Forks
3.9k
Avg merge
4d 6h
Merged PRs (30d)
34

Description

I originally mentioned that quite a while ago in the [Gaussian splat follow-ups issue](https://github.com/CesiumGS/cesium/issues/12682#issuecomment-3085200661), but it should probably be tracked in a dedicated issue:

It is currently not possible to combine Gaussian splatting tile content and **any** non-Gaussian-Splatting tile content in a single tileset.

There is a [single check](https://github.com/CesiumGS/cesium/blob/b799f639a715207d54dd7a9c9024a8f4d7bc1138/packages/engine/Source/Scene/Cesium3DTileContentFactory.js#L97) that checks whether the `3DTILES_content_gltf` extension is present and declares the `KHR_gaussian_splatting...` extensions to be used/required, and if this is the case, then it creates Gaussian splatting content from the glTF. This is wrong because it should not be necessary to declare the `3DTILES_content_gltf` extension on the tileset level. And it is wrong because this extension being declared does not mean that all glTF files contain gaussian splats. The decision has to be made based on the glTF itself - and not only based on the glTF as a whole, but based on the mesh primitives: It is necessary to _traverse_ the glTF node hierarchy, and examine each individual mesh primitive. If it declares the gaussian splatting extension, then the corresponding splats have to be taken into account. (Transformed with the global matrix of the node that the respective mesh is attached to).

Here is an archive that contains three tilesets, once with a mesh, once with splats, and once with splats and a mesh (and a corresponding sandcastle).

[Cesium Mix Splat and Mesh.zip](https://github.com/user-attachments/files/26059453/Cesium.Mix.Splat.and.Mesh.zip)

Trying to load the tileset that contains the splats and the mesh currently causes
```
TypeError: Cannot read properties of undefined (reading 'typedArray')
at GaussianSplat3DTileContent.update (GaussianSplat3DTileContent.js:685:7)
at Cesium3DTile.process (Cesium3DTile.js:2304:19)
at processTiles (Cesium3DTileset.js:2874:12)
at Cesium3DTileset.prePassesUpdate (Cesium3DTileset.js:2666:3)
at PrimitiveCollection.prePassesUpdate (PrimitiveCollection.js:457:17)
at prePassesUpdate (Scene.js:4256:20)
at tryAndCatchError (Scene.js:4362:5)
at Scene4.render (Scene.js:4435:3)
at CesiumWidget.render (CesiumWidget.js:1078:17)
at render2 (CesiumWidget.js:60:18)
```

CesiumJS version: 1.139.1 (and all before that).

Contributor guide

Open the contributing guide

Research direction

Start in packages/engine/Source/Scene/Cesium3DTileContentFactory.js at the content-type decision, then inspect GaussianSplat3DTileContent.js and the provided mixed tileset archive. Reproduce the failure with the splats-and-mesh tileset or its Sandcastle, and verify that mixed glTF primitives load without the reported TypeError while retaining their transforms.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
computer-graphics
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.