cilium / cilium/workerpool

Workerpool v2 API

Open
#45 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

kind/enhancement kind/feature
Dominant language
Go
Stars
35
Forks
6
Avg merge
9h 12m
Merged PRs (30d)
2

Description

This issue lists all breaking changes that are being considered for an eventual v2 release. The discussion started in this PR.

Breaking changes

Context-aware blocking operations

Add context.Context parameter to all blocking methods:

  • Submit(ctx context.Context, id string, f func(ctx context.Context) error) error
  • Drain(ctx context.Context) ([]Task, error)
  • Close(ctx context.Context) error

Current design ties the entire lifecycle to a single context at construction time and doesn't handle per-operation cancellation vs per-task cancellation separately.

"context should be passed to blocking functions that should be something you can cancel. That would be Submit, Drain, Close, etc." — chancez, PR #44


Constructor doesn't start goroutines

Remove automatic goroutine spawning in New() and NewWithContext(). Add explicit Run(ctx context.Context) error method.

// v1
wp := workerpool.New(10)
wp.Submit("task1", taskFunc)

// v2
wp := workerpool.New(10)
go wp.Run(context.Background())
wp.Submit(ctx, "task1", taskFunc)

"we start a goroutine in our constructor, which IMO, is an anti-pattern anyways and we shouldn't start any Goroutines, and should instead expect the caller to do it." — chancez, PR #44


Restartable worker pools

Allow calling Run() again after Close() to restart the pool. v1 pools are permanently closed.


Option validation returns errors

Make New() and NewWithContext() return (*WorkerPool, error) instead of *WorkerPool.

// v1
wp := workerpool.New(10, opts...) // panics on invalid options

// v2
wp, err := workerpool.New(10, opts...)
// TODO(v2): New/NewWithContext should return an error so that option
// validation can propagate errors instead of panicking.

Type-safe callback/drain separation

When WithResultCallback is used, don't expose Drain() method (possibly via interface or separate types).

// TODO(v2): remove ErrCallbackSet — a pool configured with
// WithResultCallback should not expose Drain.

Possible approaches: return different interfaces based on options, or separate types (WorkerPool vs WorkerPoolWithCallback).


Related v1 Improvements

  • PR #114: Added WithResultCallback to avoid unbounded result accumulation
  • PR #111: Submit rejects when parent context is done
  • PR #44: Added NewWithContext for context cancellation

Design Principles

  • Blocking APIs take a context for cancellation/timeout
  • Constructors don't start goroutines
  • Library code returns errors, not panics
  • Type safety over runtime checks

References

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading PR #44 and the rewrite prototype at pr/chancez/rewrite, then compare them with the current workerpool API. The issue lists several incompatible design directions rather than naming implementation files or tests. Done would require an agreed v2 design and coordinated implementation of its API and lifecycle changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.