cockroachdb / cockroachdb/pebble
tiering: tracking issue
- Dominant language
- Go
- Stars
- 6k
- Forks
- 584
- Avg merge
- 16h 35m
- Merged PRs (30d)
- 5
Description
This issue will track the work items related to tiering support.
- Design
- [ ] get feedback from SQL foundations
- [ ] get feedback from SQL queries
- [ ] get feedback from KV
- CockroachDB
- [ ] support configuration of tiered stores
- SQL
- [ ] tiering support in table/index descriptor ([prototype](https://github.com/cockroachdb/cockroach/commit/0de21404f7d490a6bcb0ea92961f3c0574296009))
- [ ] implement new syntax for CREATE TABLE
- [ ] populate tiering policy in span config ([prototype](https://github.com/cockroachdb/cockroach/commit/682b5b6293095fc207fbd39da79c49ab01026522))
- [ ] tiering attribute system column ([prototype](https://github.com/cockroachdb/cockroach/commit/e47660bd408819fa96d1744692af855183b3c4f4))
- [ ] tiering attribute encoding ([prototype](https://github.com/cockroachdb/cockroach/commit/ec47d8a151d2bf7f7d341db6d5fcec75c61dcaaf)]
- [ ] throttle SQL table statistics on cold data
- [ ] implement syntax for ALTER PRIMARY KEY
- [ ] implement ALTER PRIMARY KEY tiering schema change
- KV
- [ ] tiering support in span config ([prototype](https://github.com/cockroachdb/cockroach/commit/682b5b6293095fc207fbd39da79c49ab01026522)]
- Storage layer
- [ ] plumb tiering policy and attribute extraction into span policy
- Backup
- [ ] default to fewer full backups on tiered clusters #5575
- [ ] figure out how restoring a tiered cluster will work
- Pebble
- API
- [ ] extend SpanPolicy to support tiering
- [ ] extend SpanPolicy to support separating files at range boundaries
- objstorage
- [x] support cold tier directory
- [x] dual-write of metadata to both hot and cold tier
- [ ] integrate and use secondary cache with cold tier
- [ ] cold tier throttling
- sstable format
- [x] add tiering attribute and spanID columns (https://github.com/cockroachdb/pebble/issues/5466)
- [ ] add "secondary" blob reference columns
- [x] research efficient histogram/quantile sketches, come up with a usable implementation
- [x] store quantile sketch of bytes used across tiering attribute values (https://github.com/cockroachdb/pebble/issues/5566)
- compactions
- [x] unify blob value compaction strategies (https://github.com/cockroachdb/pebble/pull/5482)
- [ ] implement tiering compaction strategy
- [x] compaction iterator support for tiering metadata
- TODO
- metrics
- TODO
- policy enforcement
- [ ] background span policy enforcer #5657
Epic CRDB-48982
Jira issue: PEBBLE-1298
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.