Tracking Issue: Align the Python SDK API with the Rust (native) SDK API
- Dominant language
- Rust
- Stars
- 4.9k
- Forks
- 432
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 173
Description
### Description
This issue **works as a tracking of the process of aligning Python SDK API with the Rust core SDK API**.
**Goal:** Bring the Python SDK in line with the Rust native SDK under `core/sdk`. The table below covers both low-level and high-level APIs, with current Python status, the gap, and an adaptation difficulty estimate.
### A. Fully aligned (no work required)
| # | API | Layer |
|---|---|---|
| 1 | `connect` | Lifecycle |
| 2 | `ping` | System |
| 3 | `describe_options` | System |
| 4 | `get_user` | User |
| 5 | `get_users` | User |
| 6 | `create_user` | User |
| 7 | `delete_user` | User |
| 8 | `update_permissions` | User |
| 9 | `change_password` | User |
| 10 | `logout_user` | User |
| 11 | `get_stream` | Stream |
| 12 | `get_topic` | Topic |
| 13 | `get_topics` | Topic |
| 14 | `update_topic` | Topic |
| 15 | `delete_topic` | Topic |
| 16 | `purge_topic` | Topic |
| 17 | `get_consumer_group` | ConsumerGroup |
| 18 | `get_consumer_groups` | ConsumerGroup |
| 19 | `delete_consumer_group` | ConsumerGroup |
| 20 | `join_consumer_group` | ConsumerGroup |
| 21 | `leave_consumer_group` | ConsumerGroup |
| 22 | `send_binary_request` | Raw |
| 23 | `consumer_group()` + `IggyConsumer` | High-level |
### B. Remaining gaps (sorted by Priority, then Layer)
| # | API | Layer | Python status | Gap | Priority | Issue | PR | Difficulty |
|---|---|---|---|---|---|---|---|---|
| 1 | `get_streams` | Stream | ✅ (done) | Needs `Stream` type | High | #3520 | #3701 | Medium |
| 2 | `send_messages` | Message | ✅ (done) | `partition_id(u32)` only; missing `balanced()` / `messages_key()` | High | #3896 | #3927 | Medium |
| 3 | `poll_messages` | Message | ✅ (done) | `partition_id` mandatory `u32`; `Consumer` fixed to default; no named consumer/group | High | #3876 | #3877 | Medium |
| 4 | `store_consumer_offset` | Offset | ❌ | Needs `Consumer` type exposed | High | #3997 | TBD | Medium |
| 5 | `get_consumer_offset` | Offset | ❌ | Needs `Consumer` + `ConsumerOffsetInfo` | High | #3997 | TBD | Medium |
| 6 | `producer()` + `IggyProducer` (direct/background, retries, sharding, `create_*_if_not_exists`) | High-level | ✅ (done) | Entirely missing | High | #4019 | #4156 | Hard |
| 7 | `disconnect` | Lifecycle | ❌ | Missing | Medium | #4163 | TBD | Easy |
| 8 | `shutdown` | Lifecycle | ❌ | Missing | Medium | #4163 | TBD | Easy |
| 9 | `get_cluster_metadata` | Cluster | ❌ | Needs `ClusterMetadata` / `ClusterNode` | Medium | TBD | TBD | Medium |
| 10 | `login_user` | User | ⚠️ | Drops `IdentityInfo`, returns `None` | Medium | #4167 | TBD | Easy |
| 11 | `create_personal_access_token` | PAT | ❌ | Needs `PATExpiry` / `RawPAT` | Medium | TBD | TBD | Medium |
| 12 | `login_with_personal_access_token` | PAT | ❌ | Missing | Medium | #4167 | TBD | Easy |
| 13 | `create_stream` | Stream | ⚠️ | Drops `StreamDetails`, returns `None` | Medium | #4166 | TBD | Easy |
| 14 | `update_stream` | Stream | ✅ (done) | Needs `StreamUpdateOptions` + options | Medium | #3520 | #3701 | Medium |
| 15 | `delete_stream` | Stream | ✅ (done) | Missing | Medium | #3520 | #3701 | Easy |
| 16 | `purge_stream` | Stream | ✅ (done) | Missing | Medium | #3520 | #3701 | Easy |
| 17 | `create_topic` | Topic | ⚠️ | Drops `TopicDetails`, returns `None` | Medium | #4166 | TBD | Easy |
| 18 | `create_partitions` | Partition | ✅ (done) | Missing | Medium | #4014 | #4017 | Easy |
| 19 | `delete_partitions` | Partition | ✅ (done) | Missing | Medium | #4014 | #4017 | Easy |
| 20 | `delete_consumer_offset` | Offset | ❌ | Needs `Consumer` type exposed | Medium | #3997 | TBD | Medium |
| 21 | `create_consumer_group` | ConsumerGroup | ⚠️ | Drops `ConsumerGroupDetails`, returns `None` | Medium | #4166 | TBD | Easy |
| 22 | `IggyConsumer.shutdown()` | High-level | ❌ | Missing | Medium | #4165 | TBD | Easy |
| 23 | builder: `offset_drain_timeout` | High-level | ❌ | Missing | Medium | #4165 | TBD | Easy |
| 24 | `consumer()` (standalone consumer) | High-level | ❌ | Missing (reuse `IggyConsumer` wrapper) | Medium | TBD | TBD | Medium |
| 25 | `stream_builder` (`IggyStream*` + config) | High-level | ❌ | Missing | Medium | TBD | TBD | Medium |
| 26 | `subscribe_events` | Lifecycle | ❌ | Needs `DiagnosticEvent` stream + asyncio bridging | Low | TBD | TBD | Medium |
| 27 | `get_stats` | System | ✅ (done) | Needs `Stats` type | Low | #4016 | #4018 | Medium |
| 28 | `get_me` | System | ❌ | Needs `ClientInfoDetails` | Low | #4015 | TBD | Medium |
| 29 | `get_client` | System | ❌ | Needs `ClientInfoDetails` | Low | #4015 | TBD | Medium |
| 30 | `get_clients` | System | ❌ | Needs `ClientInfo` | Low | TBD | ¥4015 | Medium |
| 31 | `snapshot` | System | ❌ | Needs `Snapshot` / `SystemSnapshotType` | Low | TBD | TBD | Medium |
| 32 | `update_user` | User | ⚠️ | `options` hardcoded to default; `UserUpdateOptions` not exposed | Low | #4164 | #4173 | Easy |
| 33 | `get_personal_access_tokens` | PAT | ❌ | Needs `PersonalAccessTokenInfo` | Low | TBD | TBD | Medium |
| 34 | `delete_personal_access_token` | PAT | ❌ | Missing | Low | TBD | TBD | Easy |
| 35 | `delete_segments` | Segment | ❌ | Missing | Low | TBD | TBD | Easy |
| 36 | `flush_unsaved_buffer` | Message | ❌ | Missing | Low | TBD | TBD | Easy |
| 37 | `get_connection_info` | Raw | ❌ | Missing | Low | TBD | TBD | Easy |
| 38 | Client-side encryption `EncryptorKind` (AES-GCM) | Extension | ❌ | `Arc` across FFI | Low | TBD | TBD | Hard |
| 39 | Custom `Partitioner` trait | Extension | ❌ | Trait object across FFI | Low | TBD | TBD | Hard |
**Legend:** ⚠️ Supported but reduced · ❌ Missing · Priority: High / Medium / Low (by expected usage frequency) · Difficulty: Easy / Medium / Hard · PR column to be filled as each item lands.
### Out of scope — QUIC / WS / HTTP support
#2835 is tracking on this topic, when it's closed, we can disscuss futher adaption.
### Criteria
- Python SDK should behave based on the new iggy cluster implementation
- Each gap implement at least cover the unit test cases for the all the Rust SDK cases
- Detailed documentation and ues case description in `pyi` file
- For each layer/module, at least a comprehensive Python example
### Affected area / component
_No response_
### Proposed solution
_No response_
### Alternatives considered
_No response_
### Contribution
- [ ] I'm willing to submit a pull request to implement this feature
### Good first issue
- [ ] I think this could be a good first issue for a new contributor
Contributor guide
Research direction
This is a broad tracking issue rather than a single change. Start by reading the Rust SDK under core/sdk and the corresponding Python SDK declarations in the pyi files, then select one remaining gap with a defined scope. Done requires matching the Rust SDK behavior, covering the relevant unit tests, documenting the API, and adding a Python example.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100