cockroachdb / cockroachdb/cockroach
opt/indexrec: hypothetical inverted columns are assigned incorrect column IDs
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
All columns in a table are assigned consecutive columns IDs, such that if the first column in the table has column ID `c`, then the i-th column in the table has column ID `c + i`. This assumption is baked into column IDs and table IDs. For example:
https://github.com/cockroachdb/cockroach/blob/9f61a38b670e57873509568f67bb665e50cd1935/pkg/sql/opt/table_meta.go#L38-L46
When searching for index recommendations, we build hypothetical indexes and tables into a new catalogue in order to simulate optimization with indexes that don't exist. Hypothetical inverted indexes require a new inverted key column to be synthesized. This synthesized column's ordinal within its table is `1 + previous_max_table_ordinal`, but it is given a column ID that is `1 + max_column_id_in_metadata` in `opt.(*Metadata).UpdateTableMeta`: https://github.com/cockroachdb/cockroach/blob/ea781a065ab0dd5d3f8225b437458f7cb41ddab7/pkg/sql/opt/metadata.go#L1018
Therefore, the synthesized column breaks the assumption that columns within a table have consecutive column IDs. This can cause unexpected behavior and internal errors, e.g., #143826 and #144933.
I'm not sure what the best solution is. If we try to assign a column ID that is consecutive with the other columns in the table, we'll need to adjust all column IDs greater than it in the memo and metadata. I think that'll be very difficult to do correctly—those column IDs could be scattered all across different data structures in the memo. Lifting the invariant that all columns in a table have consecutive column IDs seems difficult as well.
Jira issue: CRDB-49691
Contributor guide
Assessment
This issue has not been assessed yet.