playcanvas / playcanvas/splat-transform

Operations using voxelization fail on large (volume) files

Open
#253 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
1.3k
Forks
168
Avg merge
7h 51m
Merged PRs (30d)
12

Description

When generating a voxel, or using the --filter-floaters flag, splat-transform fails during the voxelization step with files covering a large area. You can see below my example scene is parsed as 896m x 427m x 789m requiring 2.42 trillion voxels.

I'm sure I could play with the voxel/floaters settings to not have so many voxels, until it fails again.
My expectation would be that the tool caps to a maximum non-failing number of voxels - or another suggestion ?

Possibly related to #217

dev: splat-transform v2.1.0 (e1c8619)
dev: ▸ [1/2] Input temp.ply
dev:   ▸ Reading
dev:     ▸ decoding [####################] 0.513s
dev:   · 1.92M gaussians · 0 SH bands · 102.5MB
dev:   ▸ Filter floaters
dev:     · scene: 896m x 427m x 789m, grid: 17920 x 8552 x 15780 voxels (2.42T) @ 5cm
dev:     ▸ Voxelizing [####################] 5m52.574s
dev:     ✗ RangeError: Set maximum size exceeded
dev:     at Set.add (<anonymous>)
dev:     at buildBlockLookup (file://<workingDir>/node_modules/.pnpm/@playcanvas+splat-transform@2.1.0_playcanvas@2.18.1/node_modules/@playcanvas/splat-transform/dist/cli.mjs:26990:18)
dev:     at filterFloaters (file://workingDir/node_modules/.pnpm/@playcanvas+splat-transform@2.1.0_playcanvas@2.18.1/node_modules/@playcanvas/splat-transform/dist/cli.mjs:27408:24)
dev:     at async processDataTable (file://workingDir/node_modules/.pnpm/@playcanvas+splat-transform@2.1.0_playcanvas@2.18.1/node_modules/@playcanvas/splat-transform/dist/cli.mjs:29019:26)
dev:     at async main (file://workingDir/node_modules/.pnpm/@playcanvas+splat-transform@2.1.0_playcanvas@2.18.1/node_modules/@playcanvas/splat-transform/dist/cli.mjs:30136:37)
dev:     at async file://workingDir/node_modules/.pnpm/@playcanvas+splat-transform@2.1.0_playcanvas@2.18.1/node_modules/@playcanvas/splat-transform/bin/cli.mjs:5:1
dev:     failed in 6m9.507s
dev:   failed in 6m10.034s
dev: ✗ failed in 6m10.041s  [peak 8.65GB]
dev:
dev:     at genericNodeError (node:internal/errors:983:15)
dev:     at wrappedFn (node:internal/errors:537:14)
dev:     at ChildProcess.exithandler (node:child_process:417:12)
dev:     at ChildProcess.emit (node:events:519:28)
dev:     at maybeClose (node:internal/child_process:1101:16)
dev:     at ChildProcess._handle.onexit (node:internal/child_process:304:5) {
dev:   code: 1,
dev:   killed: false,
dev:   signal: null,
dev:   cmd: 'splat-transform temp.ply --filter-floaters --filter-cluster 1,0.1,0.1 temp_filtered.ply'

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with the supplied splat-transform temp.ply --filter-floaters --filter-cluster 1,0.1,0.1 temp_filtered.ply command. Trace buildBlockLookup, filterFloaters, and processDataTable from the stack trace; done means large-area voxelization no longer reaches Set maximum size exceeded and has a defined bounded behavior, though the cap policy needs clarification.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, typescript
Domain
cli, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.