vectordotdev / vectordotdev/vector

Drop_newest discards not recorded in buffer_received or buffer_discarded metrics for non blocking disk buffers

Open
#26,325 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
22.6k
Forks
2.3k
Avg merge
1d 7h
Merged PRs (30d)
146

Description

A note for the community
  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment
Problem

Events discarded from a full disk buffer configured with drop_newest do not increment buffer_received_events_total, buffer_discarded_events_total, or the corresponding byte metrics.

To reproduce bug:

  1. run vector with below config
  2. allow vector to run long enough for disk buffer to fill
  3. curl metrics endpoint, verify that
    a. vector_component_sent_events_total{component_id="load"} > vector_buffer_received_events_total{component_id="slow"}, which indicates that events discarded from the slow buffer do not increment vector_buffer_received_events_total
    b. vector_buffer_discarded_events_total and vector_component_discarded_events_total metrics do not show up, indicating no discard metric is incremented for events dropped by the buffer. (note that only the buffer discard metric is expected to exist)
Configuration
data_dir: /tmp/vector-data

sources:
  load:
    type: demo_logs
    format: json
    interval: 0.0

  metrics:
    type: internal_metrics

sinks:
  slow:
    type: blackhole
    inputs: [load]
    rate: 1
    buffer:
      type: disk
      max_size: 268435488
      when_full: drop_newest

  prometheus:
    type: prometheus_exporter
    inputs: [metrics]
    address: 127.0.0.1:9598
Version

vector 0.58.0 (aarch64-apple-darwin 2bcad9b 2026-08-26 13:37:07.557544670)

References

Related: https://github.com/vectordotdev/vector/issues/24606, mentions that the component discard metric is not incremented in this case. This issue is for the buffer discard metric.

Fixed by: #26312

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the fix referenced as #26312, then reproduce the issue with the configuration in this report and query the metrics endpoint. Done means drop_newest discards are reflected in the buffer received, buffer discarded, and corresponding byte metrics without introducing unrelated component discard metrics.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
observability
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.