test_models: add a test that fuzzes the tx messages
- Dominant language
- Python
- Stars
- 63.7k
- Forks
- 11.4k
- Avg merge
- 17h 53m
- Merged PRs (30d)
- 171
Description
Currently we compare the car states from panda and openpilot to ensure that both parse CAN messages with the same 1) validity checks and 2) correctness/no bugs. That let us fix quite a few mismatches in behavior between the two: https://github.com/commaai/openpilot/pull/30443
Now we want to fuzz the tx messages as well, to discover bugs related to controlsAllowed in panda, and general tx safety. Here is one such mismatch that we should aim to catch: https://github.com/commaai/panda/pull/1948
See the first PR linked above for rx to know where to start. Bounty is for a merged PR that adds fuzz testing for sent openpilot messages that
1) detects mismatches in tx (sent messages) logic between openpilot and panda,
1) extensible (generic, and easy to expand with more state),
2) as performant as our current fuzzy testing, and
3) covers as many cases as possible (catches a few bugs along the way, if there are any).
Contributor guide
Research direction
Start with the first linked PR and its RX fuzzing implementation to locate the existing test entry points and understand how performance is measured. Extend that approach to fuzz sent openpilot messages against panda, covering controlsAllowed and general TX safety. Done means the test detects TX mismatches, is generic to extend with more state, remains performant, and catches relevant bugs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- embedded-iot, security, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100