ConfigNamespace is slow
- Dominant language
- Python
- Stars
- 5.3k
- Forks
- 2.2k
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 74
Description
In investigating some performance issues for #12491 I found that a not-insignificant contributor is the performance of looking up values in `ConfigNamespace`.
It is actually quite slow if you look at the code, specifically for `ConfigItem.__get__`, which goes through quite a lot of steps of looking up the correct config object, reading the value from it, checking aliases, validating the value, etc.
This means that if some config option is used in a tight loop or in other common code paths it can be a bit of a drag.
For astropy.io.fits this comes up in particular for header parsing, for which there are a couple of different config options, and these get checked for *every single card* in a header when parsing its value (not a huge problem if you don't read every card in the header, but if you read a few hundred cards of each header it blows up).
I'm still analyzing exactly how much it contributes to the overall time of parsing headers, but it seems to be a significant component. I have some ideas for fixing this but still experimenting.
Contributor guide
Research direction
Start by profiling ConfigItem.__get__ during astropy.io.fits header parsing, especially the configuration lookups performed for each card. Read the investigation connected to #12491 and the existing ConfigNamespace and ConfigItem code. Done means demonstrating that configuration lookup is measurably faster in the affected parsing path, while preserving alias handling and validation behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100