makeplane / makeplane/plane

[silo] env knob to suppress Celery Producer initializer when silo is producer-only / not Celery

Open
#9,100 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
59.6k
Forks
5.8k
Avg merge
1d 22h
Merged PRs (30d)
49

Description

Summary

Silo's "Celery Producer" initializer re-declares an empty celery queue on the vhost it connects to on every AMQP reconnect, even when nothing on that vhost consumes from celery. In self-hosted topologies that isolate silo to its own RabbitMQ vhost (one vhost per Plane service), this leaves a vestigial empty queue that:

  • can't be cleaned up at the broker (silo re-declares on every reconnect),
  • pollutes rabbitmqctl export_definitions outputs consumed by backup/restore tooling, and
  • registers against generic QueueHasNoConsumers-style alerts as soon as the alert scope tightens.

Requesting an environment variable (e.g. SILO_DECLARE_CELERY=false or similar) on the silo container that skips the Celery Producer queue-declaration step when silo is producer-only / not actually feeding a Celery worker pool on its own vhost.

Reproduce

  1. Deploy Plane with one RabbitMQ vhost per service (/plane-api, /plane-silo, /plane-pi, /plane-flux); point silo at /plane-silo.
  2. rabbitmqctl delete_queue celery -p /plane-silo while silo is connected.
  3. Wait for silo's next AMQP reconnect (or roll the silo Deployment).
  4. rabbitmqctl list_queues -p /plane-silo shows celery re-declared with 0 messages, 0 consumers.

Silo's logs around the re-declare are:

Initializing Celery Producer.. ♨️
Message Queue celery connected successfully 🐇🐇🐰

Why the queue is vestigial in this topology

Celery workers live behind /plane-api's celery queue (consumed by the worker pool). /plane-silo exists purely for silo's own AMQP traffic and has no Celery consumer attached to it. The producer initializer creates the queue defensively, but on a per-service-vhost topology the queue never carries messages.

What I'd like

An env var on the silo container that disables the Celery Producer initializer's queue.declare, e.g.:

SILO_DECLARE_CELERY=false       # or DISABLE_CELERY_PRODUCER=true, naming is yours

The same knob would be useful in any topology where silo doesn't feed a Celery worker pool on its own vhost.

Workarounds tried

  • Broker-side cleanup via rabbitmqctl delete_queue -- regresses on every silo reconnect.
  • Pre-declaring the queue as a stable artifact -- still leaves the empty queue in the topology export and doesn't address the producer's intent.
  • Alert-side carve-out targeting the (/plane-silo, celery) vhost+queue pair -- shipped as a stopgap, but doesn't address the underlying re-declare loop.

Environment

  • Plane Enterprise self-hosted on Kubernetes (oke001, OKE)
  • silo image: makeplane/silo-commercial:v2.3.4
  • RabbitMQ 4.0.9 (Cluster Operator), default_queue_type=quorum cluster-wide
  • One vhost per service (/plane-api, /plane-silo, /plane-pi, /plane-flux)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Locate silo's “Celery Producer” initializer and its queue.declare step, then trace how environment settings for the container are loaded. Add the requested opt-out behavior and verify that disabling it prevents the celery queue from being re-declared while the default behavior remains unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, kubernetes, rabbitmq
Domain
backend, infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.