elastic / elastic/beats

[bug-hunter] CrowdStrike multi-feed follower does not reopen ended feed while siblings run

Open
#53,144 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

needs_team
Dominant language
Go
Stars
12.7k
Forks
5k
Avg merge
2d 1h
Merged PRs (30d)
370

Description

## Impact
When the CrowdStrike streaming input follows multiple discovered feeds concurrently, one feed ending with EOF is not re-opened if another feed stays connected. This causes silent partial ingestion: some feeds stop permanently while others continue.

## Reproduction Steps
1. Create a new test file `x-pack/filebeat/input/streaming/crowdstrike_repro_test.go` with the test in **Failing Test** below.
2. Run:
```bash
cd x-pack/filebeat/input/streaming
go test -run TestFollowSession_ReopensEndedFeedWhileOtherFeedAlive -count=1
```

## Expected vs Actual
**Expected:** when one discovered feed ends, it should be re-established (or the session should rediscover/reopen feeds) even if sibling feeds remain alive.

**Actual:** the ended feed is requested exactly once and never re-opened while another feed remains active.

Observed output:
```text
--- FAIL: TestFollowSession_ReopensEndedFeedWhileOtherFeedAlive (0.30s)
...
crowdstrike_repro_test.go:109: ended feed was not re-opened while sibling stayed alive: short feed requests=1 discover requests=1
FAIL
```

## Failing Test
```go
package streaming

import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"sync/atomic"
"testing"
"time"
)

func TestFollowSession_ReopensEndedFeedWhileOtherFeedAlive(t *testing.T) {
var shortFeedRequests atomic.Int32

shortFeed := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
shortFeedRequests.Add(1)
w.Header().Set("Content-Type", "application/json")
fmt.Fprintln(w, `{"metadata":{"eventType":"Test","offset":1},"event":{"feed":"short"}}`)
}))
defer shortFeed.Close()

longFeedBlock := make(chan struct{})
longFeed := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
flusher, ok := w.(http.Flusher)
if !ok {
t.Error("ResponseWriter does not implement http.Flusher")
return
}
w.Header().Set("Content-Type", "application/json")
fmt.Fprintln(w, `{"metadata":{"eventType":"Test","offset":2},"event":{"feed":"long"}}`)
flusher.Flush()
<-longFeedBlock
}))
t.Cleanup(func() {
close(longFeedBlock)
longFeed.Close()
})

refreshSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
}))
defer refreshSrv.Close()

discoverRequests := atomic.Int32{}
discoverSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
discoverRequests.Add(1)
w.Header().Set("Content-Type", "application/json")
resp := map[string]any{
"resources": []map[string]any{
{
"dataFeedURL": shortFeed.URL + "/firehose",
"sessionToken": map[string]any{
"token": "test-token",
"expiration": "2099-01-01T00:00:00Z",
},
"refreshActiveSessionURL": refreshSrv.URL + "/refresh",
"refreshActiveSessionInterval": 1800,
},
{
"dataFeedURL": longFeed.URL + "/firehose",
"sessionToken": map[string]any{
"token": "test-token",
"expiration": "2099-01-01T00:00:00Z",
},
"refreshActiveSessionURL": refreshSrv.URL + "/refresh",
"refreshActiveSessionInterval": 1800,
},
},
"meta": map[string]any{},
}
b, err := json.Marshal(resp)
if err != nil {
t.Errorf("failed to marshal discover response: %v", err)
return
}
_, _ = w.Write(b)
}))
defer discoverSrv.Close()

s := newTestStream(t, discoverSrv.URL, http.DefaultClient)

ctx, cancel := context.WithCancel(t.Context())
defer cancel()

done := make(chan error, 1)
go func() {
_, err := s.followSession(ctx, discoverSrv.Client(), map[string]any{})
done <- err
}()

time.Sleep(300 * time.Millisecond)
cancel()

select {
case <-done:
case <-time.After(2 * time.Second):
t.Fatal("followSession did not return after cancellation")
}

if got := shortFeedRequests.Load(); got < 2 {
t.Fatalf("ended feed was not re-opened while sibling stayed alive: short feed requests=%d discover requests=%d", got, discoverRequests.Load())
}
}
```

## Evidence
- Multi-feed workers are started concurrently and `followSession` waits for **all** workers to finish before returning: `x-pack/filebeat/input/streaming/crowdstrike.go:469-492`.
- A feed EOF is treated as success (`return nil`) and does not trigger restart/cancel: `x-pack/filebeat/input/streaming/crowdstrike.go:619-621`.
- Combined effect: if one feed ends but another stays open, rediscovery does not occur, so the ended feed remains down.
- Regression context: multi-feed behavior was introduced in `2e9f4a2d61`.

---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Bug Hunter](https://github.com/elastic/beats/actions/runs/34469575485)

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
> - [x] expires on Sep 17, 2026, 11:21 AM UTC

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with x-pack/filebeat/input/streaming/crowdstrike.go:469-492 and 619-621 to trace how concurrent feed workers handle EOF and session completion. Add the reproduction as x-pack/filebeat/input/streaming/crowdstrike_repro_test.go, then run `cd x-pack/filebeat/input/streaming && go test -run TestFollowSession_ReopensEndedFeedWhileOtherFeedAlive -count=1`. Done means the test passes and the ended feed is requested again while its sibling remains connected.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.