apache / apache/pulsar

Provide consistent concept around `partitioned topic` and `topic partition`/`internal topic`

Open
#20,622 9 comments 0 reactions 0 assignees View on GitHub
Stale type/enhancement
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

(follow up of question raised from [slack community](https://apache-pulsar.slack.com/archives/C5Z4T36F7/p1686636952934049) and discussion with @lhotari)

# Motivation

Concepts should include whatever is commonly used. But currently, there is no good definition of what "partitioned topic" and "non-partitioned topic" mean and some places `topic partition` and `internal topic` interchangeably. And there are currently two locations where we can improve on

### 1# **Topic Name API**

Let’s take look at [one of questions raised in slack community](https://apache-pulsar.slack.com/archives/C5Z4T36F7/p1686636952934049)

```java
TopicName topicName = TopicName.get("persistent://public/default/example-partition-0");
System.out.println(topicName.isPartitioned()); // true

String partitionedTopicName = topicName.getPartitionedTopicName();
TopicName.get(partitionedTopicName).isPartitioned()); // false, so it's a `PartitionedTopicName` but isn't partitioned, why? I can't understand.
```

Even though we know those who are used to Pulsar can distinguish the difference, not those without Pulsar concept. The term `partition` is somewhat overloaded with concepts.

Just to be clear,

- "**partitioned topic**" is `persistent://public/default/example` and the
- "**topic partition**" is `persistent://public/default/example-partition-0`

### **2# Documentation & Glossary**

At [terminology page of Pulsar-site](https://pulsar.apache.org/docs/3.0.x/reference-terminology/#partitioned-topic) says..

> **Partitioned Topic**
> A topic that is served by multiple brokers, which enables higher throughput.

…. which isn't consistent with the other parts of the docs and the explanations aren't sufficient. And the “other parts” refer to docs about ["topics"](https://pulsar.apache.org/docs/3.0.x/concepts-messaging/#topics) and ["partitioned topics"](https://pulsar.apache.org/docs/3.0.x/concepts-messaging/#partitioned-topics)

# Solution

### 1# Improve naming for **TopicName API**

- Deprecate `ToicName.isPartitioned()` and add new
- `TopicName.isInternallyPartitioned()` or
- `TopicName.hasInternalPartitions()` or
- `TopicName.isPartitionedType()`
- Deprecate `TopicName.getPartitionedTopicName()` and add new
- `TopicName.getBaseTopicNameOrSelf()`

### 2# Improve JavaDoc of **TopicName API**

Before there was no JavaDoc, but we can add like....
```java
/**
* @return whether the topic with (completeTopicName) has internal partitions
*/
public boolean isPartitioned() {
return partitionIndex != -1;
}
```

```java
// Before
/**
* For partitions in a topic, return the base partitioned topic name.
* Eg:
*
*/

// After
/**
* @return The base partitioned topic name if current topic is an internal partition,
* otherwise {@link #completeTopicName} as is.
*
* Eg:
*
*/
public String getPartitionedTopicName() {
if (isPartitioned()) {
return completeTopicName.substring(0, completeTopicName.lastIndexOf("-partition-"));
} else {
return completeTopicName;
}
}
```

### 3# Create a consistent glossary about concepts

Since descriptions in [Pulsar-site terminology page](https://pulsar.apache.org/docs/3.0.x/reference-terminology/) seem to keep consistency with other parts of the site, turn it into Appendix and just link to wherever sufficient description exists

# Alternatives

### Case of only improving documentation

Will still keep create confusion

# Anything else?

References

- Motivation > 2# Documentation & Glossary
- https://pulsar.apache.org/docs/3.0.x/concepts-messaging/#topics
- https://pulsar.apache.org/docs/3.0.x/concepts-messaging/#partitioned-topics
- https://pulsar.apache.org/docs/3.0.x/reference-terminology/

### Are you willing to submit a PR?

- [X] I'm willing to submit a PR!

# Link to mailing-list(dev)

https://lists.apache.org/thread/36mmvbjbm4g0b8khwmom7pjv5hr6ytxg

Contributor guide

Open the contributing guide

Research direction

Start with the TopicName API methods isPartitioned() and getPartitionedTopicName(), then compare the referenced topics, partitioned-topics, and terminology documentation pages. Done means the terminology and API naming approach are agreed, the relevant JavaDoc is consistent, and the glossary or linked documentation explains partitioned topics and topic partitions without ambiguity.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend-api-design, documentation
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.