apache / apache/pulsar

Namespace-level publish rate limits malfunction on namespace creation if not passed specific keys

Open
#16,709 1 comment 0 reactions 0 assignees View on GitHub
Stale type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

**Describe the bug**

If I try to set publish rate limits *when I create a namespace* (i.e. when making one big PUT to /namespaces/whatever), Pulsar's behavior is unintuitive and often results in rate limits that are not applied.

**To Reproduce**
1. Ensure standalone pulsar is running.
2. Using the admin REST api (not pulsar-admin) create a namespace using this subpolicy:

```
"publishMaxMessageRate": { "foo" {"publishThrottlingRateInMsg": 1,
"publishThrottlingRateInByte": 0}}
```
3. Observe that the PUT request is accepted and that `pulsar-admin namespaces policies` returns the "foo" key for that namespace's policies.
4. Attempt to publish more than 1 msg/sec to a topic in the namespace.
5. Observe that publishes greater than the specified rate limit are allowed.
6. Delete and recreate the namespace, but instead of "foo" use "standalone".
7. Repeat step 4.
8. Observe that publishes are now properly rate limited.

**Expected behavior**
1. Creating a namespace with subkeys of `publishMaxMessageRate` that will not result in namespaces in that topic having the rate limit applied should fail with an informative error indicating why the requested configuration was invalid.
2. Ideally (maybe in API v3?) the top-level key configuration would be removed, as it is [poorly documented](https://github.com/apache/pulsar/issues/16708) and doesn't make a ton of sense (and `pulsar-admin` appears to have a hidden behavior where it chooses the key to set).

**Desktop (please complete the following information):**
- OS: MacOS 12
- Pulsar 2.10.1 standalone running in Docker.

Contributor guide

Open the contributing guide

Research direction

Start with the admin REST API path for PUT /namespaces and its publishMaxMessageRate handling; compare the behavior for the "foo" and "standalone" subkeys described in the reproduction. Done means an invalid subkey configuration is rejected with an informative error instead of being accepted without enforcing the requested publish rate limit.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.