utopia-php / utopia-php/monorepo

queue: NATS stream naming is lossy and cannot be joined to the queue's own telemetry

Open
#272 0 comments 0 reactions 1 assignee View on GitHub

@levivannoort is already working on this.

Since Sep 15, 2026.

enhancement
Dominant language
PHP
Stars
3
Forks
4
Avg merge
12h 25m
Merged PRs (30d)
103

Description

Broker\Nats derives a queue's stream names by uppercasing the queue name and suffixing the dead stream (packages/queue/src/Queue/Broker/Nats.php:1219-1264):

private function workStream(Queue $queue): string   { return 'Q_' . $this->streamToken($queue->name); }
private function deadStream(Queue $queue): string   { return $this->workStream($queue) . '_DEAD'; }
private function streamToken(string $name): string  { return strtoupper(preg_replace('/[^A-Za-z0-9_-]/', '_', $name)); }
private function subjectToken(string $name): string { return strtolower(preg_replace('/[^A-Za-z0-9_-]/', '_', $name)); }

So one queue carries five spellings — worker audits, queue name v1-audits, stream Q_V1-AUDITS, dead stream Q_V1-AUDITS_DEAD, subjects q.v1-audits.* — and three of the problems below come from the two lines that disagree with each other about case.

1. The uppercasing has no precedent to appeal to. The comment at Nats.php:1221 says the scheme "mirrors JetStream's own KV_/OBJ_ streams". nats-server derives KV_my-bucket: uppercase prefix, bucket name verbatim. The Q_ prefix does mirror the convention; strtoupper does not.

2. Stream depth cannot be joined to queue depth. Server emits messaging_queue_depth{messaging_destination_name="v1-deletes"} (packages/queue/src/Queue/Server.php:257); the NATS exporter emits nats_stream_total_messages{stream_name="Q_V1-DELETES"}. Disjoint label names and disjoint values, and PromQL has no case folding outside the experimental lower()/upper(). There is no query that compares what a producer thinks it enqueued against what the stream holds — which is exactly the check a dropped-publish investigation needs. Lowercase the token and the join is a prefix strip.

3. _DEAD is a suffix in the same namespace as the name. Q_X_DEAD means both "dead stream of x" and "work stream of queue x_dead", and every separator in a real name is already _ (Q_DATABASE_DB_FRA1_SELF_HOSTED_0_0_DEAD is live). It also forces every dashboard and alert matcher into an ordered Q_V1-(.*)_DEAD-before-Q_V1-(.*) rewrite, which is where dead-letter panels and alerts have been losing five streams in appwrite-labs/cloud#5815. A QD_ prefix removes both the ambiguity and the ordering requirement.

4. The sanitiser is copy-pasted downstream. Cloud's QueueCutoverStatus.php:457 re-implements streamToken() because the helpers are private, with a comment saying so ("Mirrored rather than imported"). That mirror feeds a "safe to retire nats" verdict — i.e. "delete the destination" — and a change to the regex here desyncs it with no test failure anywhere.

Proposed fix

  • One token helper, lowercase, shared by stream and subject names, so Q_ + token is the subject token: v1-audits → stream Q_v1-audits, dead stream QD_v1-audits, subjects q.v1-audits.{normal,priority,dead}. Two names that sanitise alike still collapse to one stream, so ensure()'s identity-metadata guard keeps its current meaning.
  • Nats::workStreamName(string $queue) / Nats::deadStreamName(string $queue) as public static API, so a downstream task resolves a name instead of mirroring a regex.

Keep what the scheme gets right: the delivery tier stays out of the stream name, so a profile change is a config edit and not a rename.

This is a breaking change for anyone already running the NATS broker — existing streams keep their old names and are orphaned by the rename, so it ships as a major and lands as a drain-and-recreate. Today the only deployment on it is Appwrite Cloud staging, where all 50 streams are WorkQueue retention; after the production cutover the same change is a migration.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.