cockroachdb / cockroachdb/cockroach
sql/backfill: integrate distributed merge and ingest into legacy index backfill
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
This is a continuation of issue #158378. That was opened to add distributed merge to the index backfill. We reduced scope of that issue to focus only on integration with the declarative schema changer. This issue is opened to integrate with legacy schema changer.
To get into the path for legacy backfill you need to set this:
```
SET CLUSTER SETTING bulkio.index_backfill.distributed_merge = 'legacy';
```
We already have the first stage integrated, which takes the rows and generates the first intermediate SST.
For test coverage, we can expand existing tests with the `local-dist-merge-backfill-legacy-schema-changer` logic test config.
Jira issue: CRDB-57826
Epic CRDB-62564
Contributor guide
Research direction
Start by reading the existing distributed-merge index backfill integration and the legacy schema changer path. Enable bulkio.index_backfill.distributed_merge with the value 'legacy', then expand the existing tests using the local-dist-merge-backfill-legacy-schema-changer logic test config. Done means distributed merge and ingest are integrated through legacy index backfill with test coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, sql
- Domain
- databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100