OpenDroneMap / OpenDroneMap/ODM

Merge phase for dataset splitting doesn't seem to work

Open
#1,939 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
6.5k
Forks
1.3k
Avg merge
5d 6h
Merged PRs (30d)
4

Description

The problem has been discuss here: https://community.opendronemap.org/t/orthophotomap-merging-really-long/24700 ; others problem regarding datasets splitting has been discussed here: https://community.opendronemap.org/t/split-dataset/24299 (no splitting of PC and some corrected problems)

When processing a dataset in split mode, the map merging step is very slow and appears to be single-threaded.

System details:

  • Split mode used with 4 submodels
  • Each submodel completed successfully
  • System shows Python process using a single core at 100% during merging
  • Hardware: Nvidia 3080, i7 9900k, 96 GB RAM
  • Webodm docker for Linux and GPU

Issue:

  • Map generation (for 4 submodels) took ~20 hours — expected
  • Map merging step has been running for 16+ hours
  • CPU usage indicates single core usage only
  • Process is not stuck, but extremely slow
  • Verbose logs don't provide estimates or meaningful progress updates

Log excerpt:

[INFO] Merging all crop bounds: ['/var/www/data/...submodel_0000/...bounds.gpkg',
                                 '/var/www/data/...submodel_0001/...bounds.gpkg',
                                 '/var/www/data/...submodel_0002/...bounds.gpkg',
                                 '/var/www/data/...submodel_0003/...bounds.gpkg']
[INFO] Found 4 submodels with valid orthophotos and cutlines
[INFO] 4 valid orthophoto rasters to merge
[INFO] Output bounds: 433640.60 ... 434762.08
[INFO] Output width: 42248, height: 66783

Manual GDAL Test:

Merging the same 4 GeoTIFF files manually using GDAL completes in under 10 minutes:
gdal_merge.py -pct -ot Float32 -of GTiff -o output.tif input1.tif input2.tif input3.tif input4.tif

Questions:

  1. Why is the merging step single-threaded?
    
  2. Is it expected for this step to take 16+ hours, or is this indicative of an issue?
    
  3. Is there a way to get progress reporting or debug-level output to estimate remaining time?
    

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No source file or test is named. Start by reproducing split mode with the four submodels and compare the merge behavior with the reported GDAL command, then inspect the merge path and its logging. Done means identifying why the merge is slow or single-threaded and adding a verified improvement or actionable progress reporting.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.