matrix-org / matrix-org/complement
Autogen per-test docs for discoverability
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 99
- Forks
- 72
- Avg merge
- 4d 1h
- Merged PRs (30d)
- 8
Description
While working on importing sytests, i found myself `ctrl-shift-f`-ing on keywords to see what tests already existed to see if a sytest would fit that file, however, I think that this displays some poor test discoverability.
However, when i was documenting some sytests (and their behaviour), i came upon something like the following format;
```yaml
sytest:
- name: "Non-present room members cannot ban others"
type:
- power_level
- ban
desc:
When a user does not exist in a room, but has powerlevels there, it should not be able to ban users.
variants:
- user has never entered the room
- user has left the room
- user has been kicked while ban is sent out
- user server has been ACL-d
- name: "New federated private chats get full presence information (SYN-115)"
# ...
```
I think it would be useful to programmatically document (and maybe link) these tests, so that a quick glance can see which ones exist, what it does, what variants of behaviour it's also testing, and also in which "areas" of testing matrix it exists.
---
For a final variant of above schema, i think adding a `path` (something like `tests/csapi/rooms_state_test.go:TestRoomCreationReportsEventsToMyself`) would help auto-identify the corresponding function.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by examining the proposed YAML schema and the example entry point tests/csapi/rooms_state_test.go:TestRoomCreationReportsEventsToMyself. Determine how sytests expose names, behavior, variants, and testing areas for automated documentation. Done means generated documentation can list those details and link each entry to its test path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, yaml
- Domain
- documentation, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100