ByteDance-Seed / ByteDance-Seed/Depth-Anything-3

Can't train 3DGS on export_to_colmap results

Open
#117 11 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
6.3k
Forks
702
PR merge metrics
No merged PRs in 30d

Description

I exported predictions to Colmap format using `export_to_colmap` function: it generates `points3D.bin`, `images.bin`, `frames.bin`, `cameras.bin` and `rigs.bin`. I note here that for 300 images it generates 20M points with a confidence percentile of 70.

Next, I tried to run `train.py` from original gaussian splatting repo (`graphdeco-inria`) but I consistently get this error:

```
Converting point3d.bin to .ply, will happen only the first time you open the scene. [28/11 23:25:12]
Loading Training Cameras [28/11 23:26:20]
Loading Test Cameras [28/11 23:26:48]
Number of points at initialisation : 12827810 [28/11 23:26:49]

[gpu-health] [WARN] GPU-a2edc31e-63f6-ebc5-f156-ed40ee9f221b: XID: NVRM: Xid (PCI:0000:ca:00): 31, pid=1907695, name=exe, Ch 00000008, intr 00000000. MMU Fault: ENGINE GRAPHICS GPC11 GPCCLIENT_T1_0 faulted @ 0x2b0d_7b7bf000. Fault is of type FAULT_PDE ACCESS_TYPE_VIRT_READ
Training progress: 0%| | 0/7000 [00:00
training(lp.extract(args), op.extract(args), pp.extract(args), args.test_iterations, args.save_iterations, args.checkpoint_iterations, args.start_checkpoint, args.debug_from)
File "/app/gaussian-splatting/train.py", line 142, in training
loss.backward()
File "/opt/conda/envs/torch/lib/python3.10/site-packages/torch/_tensor.py", line 521, in backward
torch.autograd.backward(
File "/opt/conda/envs/torch/lib/python3.10/site-packages/torch/autograd/__init__.py", line 289, in backward
_engine_run_backward(
File "/opt/conda/envs/torch/lib/python3.10/site-packages/torch/autograd/graph.py", line 769, in _engine_run_backward
return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

Training progress: 0%| | 0/7000 [00:00

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the export_to_colmap output files—points3D.bin, images.bin, frames.bin, cameras.bin, and rigs.bin—and reproduce the failure with graphdeco-inria's train.py. Inspect the point-count initialization and the backward-pass CUDA error. Done means documenting whether the generated point count or COLMAP output is unsupported and identifying the required correction or limit.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.