Understand impact of cpu-architecture specific floating-point optimizations on join plans.
- Dominant language
- Go
- Stars
- 24.4k
- Forks
- 873
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 120
Description
There are a small number of join plans that, as of the current main branch ([2f7e3c8](https://github.com/dolthub/go-mysql-server/pull/3335/commits/2f7e3c883a6c0807df99b6d9128fdfba22376be8)), produce different plans on AMD vs ARM processors.
These are the ImdbQueryPlan tests. The affected tests in this test suite are currently disabled.
As best as I currently understand it, here is why this is happening:
1. There are multiple possible join plans for these queries that have theoretically identical costs
2. Due to floating-point errors, one of these plans will actually have a slightly lower cost than the other (by a single epsilon), resulting in it being preferred over the other.
3. The Go compiler is allowed to emit instructions that produce less floating point error than would be implied by a strict implementation of IEE-754. (See https://github.com/golang/go/issues/17895 for more context)
4. The exact behavior of these instructions is CPU dependent
As a result, it becomes possible for Dolt running on ARM processors to pick one plan, and Dolt running on AMD processors to pick another.
In practice, this likely doesn't matter much: if two joins literally have the exact same cost estimate, it's extremely likely the plans are effectively identical, ultimately represent the same physical operation, and have the same performance characteristics in practice.
But it matters for tests, and we should consider how to address this.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the disabled ImdbQueryPlan tests and compare the plans produced on AMD and ARM processors, using commit 2f7e3c8 as context. Determine how floating-point cost differences affect plan selection and document the impact; the work is done when there is an agreed approach for addressing the architecture-dependent test results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100