General optimisations for dict lookup/insert performance
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 35.9k
- PR merge metrics
- PR metrics pending
Description
Proposal:
There's room for tiny refactorings in the Objects/dictobject.c that may allow speedups, for example via better CPU cache behavior, compiler hints, cheap checks & fast exits for common cases. In my guess-timation, they may bring about upto 10% overall lookup/insert performance gains. That's more of an un-scientific intuition at best right now, but I'm laying some ideas out here:
-
compare_unicode_unicode(): can benefit from pointer-equality and branch prediction, and reducing unnecessary function calls -
compare_generic(): can use early rejections on hash-mismatch, avoidingRichCompareBoolcalls -
unicodekeys_lookup_unicode(): pointer-equality for the common case can precededo_lookupwhich today is invoked everytime for all cases -
find_empty_slot(): early termination / separate path for first iteration can speed up insertions, especially for lightly-populated dicts -
build_indices_generic / build_indices_unicode (): breaking out the first-probe success case may speed up dict resize operations
I welcome feedback on how to raise this proposal differently, but otherwise I'd like to work on these ideas and quantify them. I feel modern CPUs have deep pipelines and these optimizations may help us avoid some cache-thrashing. These changes are purely performance oriented and don't change functionality / API / ABI and should be thread-safe too.
Has this already been discussed elsewhere?
This is a minor feature, which does not need previous discussion elsewhere
Links to previous discussion of this feature:
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in Objects/dictobject.c and inspect the named lookup, comparison, empty-slot, and index-building functions. Quantify each proposed optimization with performance measurements, while confirming that dictionary functionality, API, ABI, and thread-safety behavior remain unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, python
- Domain
- performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100