transparency-dev / transparency-dev/tessera
Feature: support resource sharing for sharded logs
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 238
- Forks
- 56
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 35
Description
As part of #574 I was documenting how to shard logs. The current solution is simply to bring up a whole new stack, including DB instances, buckets, VMs, binaries, load balancing rules, etc.
At least some of this could be reused. e.g. in GCP, a single Spanner instance could host multiple databases (one for each log). #267 covers this aspect. It's also logically reasonable that a single personality could route through to multiple log instances. Technically this is possible at the moment, but the monitoring being added as part of #570 will not disambiguate which log the counters. For some of the gauges that are exported, running multiple log appenders in the same process could totally break the monitoring dashboards as the two appender monitoring tasks stomp on each other.
This issue is raised to allow us to collect a demand signal, and ideas from the community on the features expected.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the context in #574, the Spanner-related work in #267, and monitoring changes in #570. Gather the expected resource-sharing and multi-log monitoring requirements; done means the community has turned this demand signal into an agreed, actionable scope.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- gcp, go
- Domain
- backend, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100