liquidmetal-dev / liquidmetal-dev/flintlock
e2e: Check more physical outcomes
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 1.5k
- Forks
- 73
- Avg merge
- 17h 24m
- Merged PRs (30d)
- 30
Description
Right now the e2es perform simple CRUD (more like CRD) operations and check a couple of observable state things outside of flintlock:
- Checks the socket exists
- Reads the pidfile and checks the pid is running
But this is not a fool-proof test, and mvms can get into the state where both those (and other physical) things are fine, and flintlock records the status as created successfully, and yet something is broken and the mvm is not running.
A recent instance of this is where the root volume was misconfigured and therefore the mvm could not boot properly, but everything else looked "fine" so the e2es did not fail.
This example is a tricky one to test at the e2e (or any) level, as you don't know something is wrong until firecracker tries to use it. And short of making the tests parse the logs (I am not doing that), it is very hard to see the "evidence" of failure.
Options in this case are:
- We have a test fixture which checks that the fl/fc config returned by the created mvm aligns with what we expect (this is a very superficial test, but a start)
- We actually do parse the fl logs for failures (i really don't want to do this as it is very brittle)
- We watch the event stream for bad events (ditto above, but slightly less crap)
- We ensure that `/x/y/x/firecracker.stderr` is empty
- We use the containerd client directly to check things
- We use the created firecracker cfg to start an mvm directly with firecracker itself (this feels extremely weird to do at this level because isn't creating mvms what fl is for? like it shouldn't mark an mvm as Created if it blatantly isn't)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the existing e2e CRUD checks for socket and pidfile assertions, then compare them with the created MVM configuration, event stream, firecracker.stderr, and containerd state. Decide which observable outcome is reliable enough for this test layer; done means an MVM that fails to boot or run causes the e2e test to fail without parsing logs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100