apache / apache/iggy

BDD Test Coverage Roadmap

Open
#3,609 2 comments 0 reactions 0 assignees View on GitHub
test
Dominant language
Rust
Stars
4.9k
Forks
432
Avg merge
2d 10h
Merged PRs (30d)
173

Description

# BDD Test Coverage Roadmap

Cross-SDK BDD test implementation progress. Shared `.feature` files live in `bdd/scenarios/`; each SDK implements step definitions in its own directory. Related: #2148.

Rust already has extensive integration test coverage for most of these features under `core/integration/tests/server/scenarios/`. These serve as the reference implementation. The goal is to promote them into shared Gherkin `.feature` files and implement step definitions across all SDKs.

The BDD tests should expand coverage beyond what the Rust integration tests already verify. The integration tests check internal server state (on-disk files, in-memory structures). The BDD tests verify end-to-end client-visible behavior through the full stack (SDK → wire protocol → server) and should systematically add scenarios the integration tests don't cover:
- Cross-resource isolation (does purging stream A break stream B?)
- Error cases (non-existing IDs, invalid inputs)
- Multi-resource operations (purge stream vs purge topic)
- Repeated operations after mutation (send → purge → send → poll)

## Master Progress Table

Each SDK checkbox means: `.feature` file exists in `bdd/scenarios/` AND that SDK's step definitions pass in CI.

| Feature | Feature file | Rust | Go | C# | Python | PHP | Java | Node | C++ |
|---------|:------------:|:----:|:--:|:--:|:-----:|:--:|:--:|:---:|:---:|
| [`basic_messaging`](#basic_messaging) | ✅ |✅ | ✅ | ✅ | ✅ | ✅| ✅ | ✅ | ✅ |
| [`leader_redirection`](#leader_redirection) | ✅ | ✅ | ✅ | ✅ | [ ] | [ ] | [ ] | [ ] |[ ] |
| [`stream_crud`](#stream_crud) | ✅ | [ ] | [ ] | [ ] | [ ] | [ ] | ✅ | [ ] |[ ] |
| [`topic_crud`](#topic_crud) | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] |[ ] |
| [`consumer_group`](#consumer_group) | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] |[ ] |
| [`consumer_offset`](#consumer_offset) | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] |[ ] |
| [`user_management`](#user_management) | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] |[ ] |
| [`personal_access_token`](#personal_access_token) | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] |[ ] |
| [`message_operations`](#message_operations) | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] |[ ] |
| [`partition_management`](#partition_management) | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] |[ ] |
| [`system_operations`](#system_operations) | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] |[ ] |
| [`stream_topic_purge`](#stream_topic_purge) | 🚧 | 🚧 | 🚧 | [ ] | [ ] | [ ] | [ ] | [ ] |[ ] |
| [`snapshot`](#snapshot) | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] |[ ] |
|[`rawcommend`](#rawcommand) | ✅ |✅ | ✅ | ✅ | ✅ | ✅| ✅ | ✅ | ✅ |
---

## basic_messaging

Create a stream, topic, send 10 messages, poll them back, verify offsets and payloads.

**1 scenario** in `bdd/scenarios/basic_messaging.feature`

| SDK | File | PR |
|-----|------|:---:|
| Rust | `bdd/rust/tests/basic_messaging.rs` | #1842 |
| Go | `bdd/go/tests/basic_messaging.go` | #1922 |
| C# | `foreign/csharp/Iggy_SDK.Tests.BDD/StepDefinitions/BasicMessagingOperationsSteps.cs` | #1980 |
| Python | `bdd/python/tests/test_basic_messaging.py` | #1909|
| PHP | `bdd/php/tests/BasicMessagingFeatureTest.php` | #3410 |
| Java | `bdd/java/src/test/java/org/apache/iggy/bdd/BasicMessagingSteps.java` | #2535 |
| Node.js | `foreign/node/src/bdd/` | #1862 |
| C++ | `bdd/cpp/features/step_definitions/messaging_steps.cpp` | #3554 |

## leader_redirection

Client redirects from follower to leader; direct leader connect; single-server fallback; multiple clients converge to same leader.

**4 scenarios** in `bdd/scenarios/leader_redirection.feature`. **Requires cluster compose.**

| SDK | File | PR |
|-----|------|:---:|
| Rust | `bdd/rust/tests/leader_redirection.rs` | #2359 |
| Go | `bdd/go/tests/leader_redirection.go` | #2580|
| C# | `foreign/csharp/Iggy_SDK.Tests.BDD/StepDefinitions/LeaderRedirectionSteps.cs` | #3193 |
| Python | — | |
| PHP | — | |
| Java | — | |
| Node.js | — | |
| C++ | — | |

---

## stream_crud

Create, get-by-id, list-all, update, delete streams.

- **Rust reference:** `core/integration/tests/cli/stream/test_stream_{create,delete,get,list,update}_command.rs`

| SDK | File | PR |
|-----|------|:---:|
| Rust | bdd/rust/tests/steps/streams.rs| #3875 |
| Go | — | |
| C# | — | |
| Python | — | |
| PHP | — | |
| Java | [bdd/java/src/test/java/org/apache/iggy/bdd/BasicMessagingSteps.java| #3875 |
| Node.js | — | |
| C++ | — | |

## topic_crud

Create, get-by-id, list-all, update, delete topics within a stream.

- **Rust reference:** `core/integration/tests/cli/topic/test_topic_{create,delete,get,list,update}_command.rs`

| SDK | File | PR |
|-----|------|:---:|
| Rust | — | |
| Go | — | |
| C# | — | |
| Python | — | |
| PHP | — | |
| Java | — | |
| Node.js | — | |
| C++ | — | |

## consumer_group

Create, get-by-id, list-all, delete consumer groups. Join and leave consumer groups.

- **Rust reference:** `core/integration/tests/server/scenarios/consumer_group_join_scenario.rs`, `consumer_group_duplicate_name_create_scenario.rs`, `consumer_group_with_single_client_polling_messages_scenario.rs`, `consumer_group_with_multiple_clients_polling_messages_scenario.rs`

| SDK | File | PR |
|-----|------|:---:|
| Rust | — | |
| Go | — | |
| C# | — | |
| Python | — | |
| PHP | — | |
| Java | — | |
| Node.js | — | |
| C++ | — | |

## consumer_offset

Store/retrieve consumer offsets. Verify auto-commit and offset correctness.

- **Rust reference:** `core/integration/tests/server/scenarios/offset_scenario.rs`, `invalid_consumer_offset_scenario.rs`, `consumer_group_auto_commit_reconnection_scenario.rs`, `consumer_group_offset_cleanup_scenario.rs`

| SDK | File | PR |
|-----|------|:---:|
| Rust | — | |
| Go | — | |
| C# | — | |
| Python | — | |
| PHP | — | |
| Java | — | |
| Node.js | — | |
| C++ | — | |

## user_management

Create, update, delete users. Change password. Assign and verify permissions.

- **Rust reference:** `core/integration/tests/server/scenarios/user_scenario.rs`, `permissions_scenario.rs`

| SDK | File | PR |
|-----|------|:---:|
| Rust | — | |
| Go | — | |
| C# | — | |
| Python | — | |
| PHP | — | |
| Java | — | |
| Node.js | — | |
| C++ | — | |

## personal_access_token

Create with expiry, list, revoke PATs. Authenticate with a token. Verify expiry and scoping.

- **Rust reference:** `core/integration/tests/server/scenarios/cross_protocol_pat_scenario.rs`

| SDK | File | PR |
|-----|------|:---:|
| Rust | — | |
| Go | — | |
| C# | — | |
| Python | — | |
| PHP | — | |
| Java | — | |
| Node.js | — | |
| C++ | — | |

## message_operations

Send messages with custom headers. Poll with offset/consumer-group semantics. Verify header round-trip. Verify header size limit (≤ 64 KB).

- **Rust reference:** `core/integration/tests/server/scenarios/message_headers_scenario.rs`, `message_size_scenario.rs`, `create_message_payload.rs`, `single_message_per_batch_scenario.rs`

| SDK | File | PR |
|-----|------|:---:|
| Rust | — | |
| Go | — | |
| C# | — | |
| Python | — | |
| PHP | — | |
| Java | — | |
| Node.js | — | |
| C++ | — | |

## partition_management

Create and delete partitions on an existing topic. Verify partition count updates.

- **Rust reference:** `core/integration/tests/cli/partition/test_partition_{create,delete}_command.rs`

| SDK | File | PR |
|-----|------|:---:|
| Rust | — | |
| Go | — | |
| C# | — | |
| Python | — | |
| PHP | — | |
| Java | — | |
| Node.js | — | |
| C++ | — | |

## system_operations

Fetch server stats (PID, memory, CPU, counts). Ping. Fetch client info.

- **Rust reference:** `core/integration/tests/server/scenarios/system_scenario.rs`, `core/integration/tests/cli/system/test_{ping,stats,snapshot}_*.rs`

| SDK | File | PR |
|-----|------|:---:|
| Rust | — | |
| Go | — | |
| C# | — | |
| Python | — | |
| PHP | — | |
| Java | — | |
| Node.js | — | |
| C++ | — | |

## stream_topic_purge

Purge stream (all topics); purge single topic; verify isolation between streams; verify isolation between topics; error on non-existing stream; error on non-existing topic.

**6 scenarios** in `bdd/scenarios/stream_topic_purge.feature`

- **Rust reference:** `core/integration/tests/server/scenarios/purge_delete_scenario.rs` (run_purge_topic)

| SDK | File | PR |
|-----|------|:---:|
| Rust | `bdd/rust/tests/stream_topic_purge.rs` | #3608 |
| Go | `bdd/go/tests/stream_topic_purge.go` | #3608 |
| C# | — | |
| Python | — | |
| PHP | — | |
| Java | — | |
| Node.js | — | |
| C++ | — | |

## snapshot

Create and restore file-based snapshots. Verify data integrity after restore.

- **Rust reference:** `core/integration/tests/server/scenarios/snapshot_scenario.rs`

## raw_command

Send a command code with a payload directly, so commands that have no typed SDK method can be called. Verify round-trips for
known codes, rejection for unknown and session-control codes.

**3 scenarios** in `bdd/scenarios/raw_command.feature`

| SDK | File | PR |
|-----|------|:---:|
| Rust | `bdd/rust/tests/raw_command.rs` | #3691 |
| Go | `bdd/go/tests/raw_command.go` | #3691 |
| C# | `foreign/csharp/Iggy_SDK.Tests.BDD/StepDefinitions/RawCommandSteps.cs` | #3691 |
| Python | `bdd/python/tests/test_raw_command.py` | #3691 |
| PHP | `bdd/php/tests/RawCommandFeatureTest.php` | #3691 |
| Java | `bdd/java/src/test/java/org/apache/iggy/bdd/BasicMessagingSteps.java` | #3691 |
| Node.js | `foreign/node/src/bdd/raw.ts` | #3691 |
| C++ | `bdd/cpp/features/step_definitions/raw_command_steps.cpp` | #3691 |
---

## rawcommand

Contributor guide

Open the contributing guide

Research direction

Choose one unchecked feature/SDK pair from the roadmap, then read its shared scenario in bdd/scenarios/ and the corresponding Rust reference under core/integration/tests. Check existing SDK BDD tests and run that SDK's BDD suite first. Done means the shared .feature file exists, the SDK step definitions pass in CI, and the progress table can be updated.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, csharp, go, java, node.js, php, python, rust
Domain
testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.