OpenTTD / OpenTTD/nml

Sprite Encoding/Cache/Palette Check performance is poor due to generic.find_file

Open
#440 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
59
Forks
46
Avg merge
11d 15h
Merged PRs (30d)
3

Description

Pointing procmon at nmlc on windows reveals that before every actual read of a sprite png file, there's a full enumeration of the directory that sprite contains as nmlc attempts to normalise file case and check for potential duplicates/name collisions.

Obviously on windows this is really a nonsense, but it does impact performance in grfs that have a large number of input files in a directory. In my tests (boring hardware, large grf) 75% of the process time was apparently spent under find_file.

I think the obvious answer here is to simplify find_file to not handle case conversion at all (demand authors write better nml). Everything else is case sensitive.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at generic.find_file, which nmlc calls before reading sprite PNG files, and trace how it normalizes case and checks duplicate or name collisions. Use procmon with a large GRF to compare directory enumeration before and after the change; done means avoiding the unnecessary enumeration while preserving the intended case-sensitive file behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
compilers, performance, tooling
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.