Allow per-function BYOO collector resource and log sampling overrides
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 218
- Forks
- 72
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 427
Description
Is this related to a problem? Please describe.
BYOO collector behavior is currently governed by platform or operator-level defaults. Those defaults cannot account for the different telemetry profiles of individual functions:
- A high-throughput function may need more CPU or memory for its injected BYOO collector to avoid backpressure, dropped records, or collector restarts.
- A high-volume function may need to sample informational logs before export to control ingestion volume and cost.
- A low-volume or debugging workload may need to retain all logs without forcing the same policy on every function in the account or cluster.
Operators can work toward global BYOO resource configuration through #273, but end users still need a supported, function-scoped configuration surface. Requiring application changes or operator intervention for every workload does not provide sufficient tenant-level control.
Describe the solution you would like
Add supported per-function BYOO collector overrides in two areas:
1. Collector resource overrides
Allow a function owner to set CPU and memory requests and limits for the BYOO collector associated with that function. These settings should affect only the selected function or function version and should work for both container-based and Helm-based functions.
2. Log sampling overrides
Allow a function owner to configure log sampling for that function before logs are exported by the BYOO collector. At minimum, the API should support enabling or disabling sampling and selecting a sampling rate. The sampling behavior and stage in the logs pipeline should be documented so users can reason about which records are retained.
The configuration could be exposed through the existing function create/update API, CLI, or function specification. The exact API shape is open for design, but it should provide these properties:
- Function-scoped settings do not alter account-wide or cluster-wide defaults.
- Omitting an override preserves the current platform defaults and existing behavior.
- Invalid resource quantities and sampling rates are rejected with actionable validation errors.
- Platform-defined minimums and maximums can be enforced as guardrails.
- The effective configuration is observable to the function owner for troubleshooting.
- Container and Helm functions have equivalent behavior.
Suggested acceptance criteria:
- A function can set BYOO collector CPU and memory requests and limits independently of other functions.
- A function can set a log sampling rate independently of other functions.
- A function with no overrides continues to use the existing defaults.
- Updating or creating a new function version applies the requested settings only to that version according to the established function lifecycle.
- End-to-end tests verify resource propagation and log sampling for both container and Helm function paths.
- Public documentation includes configuration examples, defaults, validation rules, and sampling semantics.
Describe alternatives you have considered
- Configure collector resources globally through the NVCA/operator settings tracked in #273. This helps operators tune a deployment but cannot accommodate different functions with very different telemetry volume.
- Configure log sampling in each application SDK. This requires every workload to implement and maintain equivalent sampling logic, and it does not help with logs collected outside the application instrumentation path.
- Patch generated workloads or collector configuration. This is unsupported, difficult to automate safely, and can be overwritten during reconciliation.
Additional context
- #273 tracks operator-level BYOO log chunking and telemetry resource configuration. This request is specifically for end-user, per-function overrides and should complement those defaults rather than replace them.
- #16 tracks general BYOO support for self-managed deployments.
- #27 lists BYOO under the Q3 observability roadmap.
By submitting this issue, you agree to follow our code of conduct and our contributing guidelines.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the existing function create/update API, CLI, and function specification, then review the operator-level defaults tracked in #273. Define how per-function and per-version resource and sampling overrides propagate through both container and Helm paths. Done includes validation, effective-configuration visibility, end-to-end coverage for both paths, and public documentation of defaults and sampling semantics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, helm, kubernetes
- Domain
- backend-api-design, cloud, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100