dolthub / dolthub/dolt

Understand impact of cpu-architecture specific floating-point optimizations on join plans.

Open
#10,211 0 comments 0 reactions 0 assignees View on GitHub
analyzer enhancement question
Dominant language
Go
Stars
24.4k
Forks
873
Avg merge
1d 8h
Merged PRs (30d)
120

Description

There are a small number of join plans that, as of the current main branch ([2f7e3c8](https://github.com/dolthub/go-mysql-server/pull/3335/commits/2f7e3c883a6c0807df99b6d9128fdfba22376be8)), produce different plans on AMD vs ARM processors.

These are the ImdbQueryPlan tests. The affected tests in this test suite are currently disabled.

As best as I currently understand it, here is why this is happening:
1. There are multiple possible join plans for these queries that have theoretically identical costs
2. Due to floating-point errors, one of these plans will actually have a slightly lower cost than the other (by a single epsilon), resulting in it being preferred over the other.
3. The Go compiler is allowed to emit instructions that produce less floating point error than would be implied by a strict implementation of IEE-754. (See https://github.com/golang/go/issues/17895 for more context)
4. The exact behavior of these instructions is CPU dependent

As a result, it becomes possible for Dolt running on ARM processors to pick one plan, and Dolt running on AMD processors to pick another.

In practice, this likely doesn't matter much: if two joins literally have the exact same cost estimate, it's extremely likely the plans are effectively identical, ultimately represent the same physical operation, and have the same performance characteristics in practice.

But it matters for tests, and we should consider how to address this.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the disabled ImdbQueryPlan tests and compare the plans produced on AMD and ARM processors, using commit 2f7e3c8 as context. Determine how floating-point cost differences affect plan selection and document the impact; the work is done when there is an agreed approach for addressing the architecture-dependent test results.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.