cockroachdb / cockroachdb/cockroach

sql/backfill: integrate distributed merge and ingest into legacy index backfill

Open
#159,397 0 comments 0 reactions 0 assignees View on GitHub
C-enhancement T-sql-foundations
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

This is a continuation of issue #158378. That was opened to add distributed merge to the index backfill. We reduced scope of that issue to focus only on integration with the declarative schema changer. This issue is opened to integrate with legacy schema changer.

To get into the path for legacy backfill you need to set this:
```
SET CLUSTER SETTING bulkio.index_backfill.distributed_merge = 'legacy';
```

We already have the first stage integrated, which takes the rows and generates the first intermediate SST.

For test coverage, we can expand existing tests with the `local-dist-merge-backfill-legacy-schema-changer` logic test config.

Jira issue: CRDB-57826

Epic CRDB-62564

Contributor guide

Open the contributing guide

Research direction

Start by reading the existing distributed-merge index backfill integration and the legacy schema changer path. Enable bulkio.index_backfill.distributed_merge with the value 'legacy', then expand the existing tests using the local-dist-merge-backfill-legacy-schema-changer logic test config. Done means distributed merge and ingest are integrated through legacy index backfill with test coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.