transparency-dev / transparency-dev/tessera

Feature: support resource sharing for sharded logs

Open
#589 0 comments 9 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Go
Stars
238
Forks
56
Avg merge
1d 2h
Merged PRs (30d)
35

Description

As part of #574 I was documenting how to shard logs. The current solution is simply to bring up a whole new stack, including DB instances, buckets, VMs, binaries, load balancing rules, etc.

At least some of this could be reused. e.g. in GCP, a single Spanner instance could host multiple databases (one for each log). #267 covers this aspect. It's also logically reasonable that a single personality could route through to multiple log instances. Technically this is possible at the moment, but the monitoring being added as part of #570 will not disambiguate which log the counters. For some of the gauges that are exported, running multiple log appenders in the same process could totally break the monitoring dashboards as the two appender monitoring tasks stomp on each other.

This issue is raised to allow us to collect a demand signal, and ideas from the community on the features expected.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the context in #574, the Spanner-related work in #267, and monitoring changes in #570. Gather the expected resource-sharing and multi-log monitoring requirements; done means the community has turned this demand signal into an agreed, actionable scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
gcp, go
Domain
backend, observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.