Namespace-level publish rate limits malfunction on namespace creation if not passed specific keys
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
**Describe the bug**
If I try to set publish rate limits *when I create a namespace* (i.e. when making one big PUT to /namespaces/whatever), Pulsar's behavior is unintuitive and often results in rate limits that are not applied.
**To Reproduce**
1. Ensure standalone pulsar is running.
2. Using the admin REST api (not pulsar-admin) create a namespace using this subpolicy:
```
"publishMaxMessageRate": { "foo" {"publishThrottlingRateInMsg": 1,
"publishThrottlingRateInByte": 0}}
```
3. Observe that the PUT request is accepted and that `pulsar-admin namespaces policies` returns the "foo" key for that namespace's policies.
4. Attempt to publish more than 1 msg/sec to a topic in the namespace.
5. Observe that publishes greater than the specified rate limit are allowed.
6. Delete and recreate the namespace, but instead of "foo" use "standalone".
7. Repeat step 4.
8. Observe that publishes are now properly rate limited.
**Expected behavior**
1. Creating a namespace with subkeys of `publishMaxMessageRate` that will not result in namespaces in that topic having the rate limit applied should fail with an informative error indicating why the requested configuration was invalid.
2. Ideally (maybe in API v3?) the top-level key configuration would be removed, as it is [poorly documented](https://github.com/apache/pulsar/issues/16708) and doesn't make a ton of sense (and `pulsar-admin` appears to have a hidden behavior where it chooses the key to set).
**Desktop (please complete the following information):**
- OS: MacOS 12
- Pulsar 2.10.1 standalone running in Docker.
Contributor guide
Research direction
Start with the admin REST API path for PUT /namespaces and its publishMaxMessageRate handling; compare the behavior for the "foo" and "standalone" subkeys described in the reproduction. Done means an invalid subkey configuration is rejected with an informative error instead of being accepted without enforcing the requested publish rate limit.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100