Enhance the channel buffer to enhance the concurrency (currently working without buffered channels)
- Dominant language
- Go
- Stars
- 58
- Forks
- 55
- Avg merge
- 9h 9m
- Merged PRs (30d)
- 424
Description
The whole vulnerability flow is divided using channels.
Every component works on an input channel and outputs to the next channel.
The components run as `goroutines` in parallel (currently with a scale of 1, preferably scale them according to available cores).
These channels are non-buffered as of right now, which means that this flow happens in a serial manner.
We would like to consider and change that behavior in order to make sure to parallelize some of the work.
This can be done easily by converting the channels to buffered channels and adding a loop for every component that we want to scale likewise:
```go
func (f *VulnerabilityWorker) Run(ctx context.Context) {
// TODO: Handle deletion of snapshots
f.log.Info("Starting VulnerabilityWorker.work")
for {
select {
case <-ctx.Done():
f.log.Info("VulnerabilityWorker.work context canceled")
return
default:
f.wg.Add(1)
go func() {
defer f.wg.Done()
f.fetcher.FetchInstances(ctx)
f.log.Info("VulnerabilityWorker.work FetchInstances finished")
}()
*FOR LOOP TO SCALE*
f.wg.Add(1)
go func() {
defer f.wg.Done()
f.replicator.SnapshotInstance(ctx, f.fetcher.GetChan())
f.log.Info("VulnerabilityWorker.work SnapshotInstance finished")
}()
f.wg.Add(1)
go func() {
defer f.wg.Done()
f.verifier.VerifySnapshot(ctx, f.replicator.GetChan())
f.log.Info("VulnerabilityWorker.work VerifySnapshot finished")
}()
f.wg.Add(1)
go func() {
defer f.wg.Done()
f.evaluator.EvaluateSnapshot(ctx, f.verifier.GetChan())
f.log.Info("VulnerabilityWorker.work EvaluateSnapshot finished")
}()
*FOR LOOP TO SCALE END*
f.log.Info("VulnerabilityWorker.work waiting on workers")
f.wg.Wait()
f.log.Info("VulnerabilityWorker.work finished waiting on workers")
return
}
}
}
```
This ticket depends on the progress and implementation of the following ticket:
- https://github.com/elastic/cloudbeat/issues/691
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.