Report device health metrics for AWS leased devices
Open
@yangw-dev is already working on this.
Since Apr 14, 2025.
module: benchmark
triaged
- Dominant language
- Python
- Stars
- 5k
- Forks
- 1.2k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 581
Description
🚀 The feature, motivation and pitch
Per the discussion with AWS, they don't have API to either collect or query the device health metrics we care regarding to provide stable benchmark metrics from run-to-run. So we will need to build this muscle ourselves and making it a standard process that running both prior and post to an actual on-device benchmark run.
Alternatives
AWS team suggests this is doable via test spec.
Additional context
This would be useful to root cause the metrics instability issue caused by devices regardless the device is leased from AWS public pool, private pool or even user-owned device.
RFC (Optional)
No response
cc @huydhn @kirklandsign @shoumikhin
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.