[polymetis] Simplify hardware benchmarking

## Type of Issue

It's a bit onerous to actually run the benchmark on hardware, with the result that it's less likely it'll happen. We should be (1) more strict about merging PRs without running the hardware CI, (2) reduce friction for running the hardware CI itself.

cc @0wu

Select the type of issue:
- [ ] Bug report (to report a bug)
- [x] Feature request (to request an additional feature)
- [ ] Tracker (I am just using this as a tracker)
- [ ] Refactor request
- [ ] Documentation Ask

## Description

Detailed description of the requested feature/documentation ask/bug report.

## Current Behavior

Description of current behavior or functionality.

## Expected Behavior

Description of the expected behavior or functionality. In case of feature request, please add input and expected output.

## Steps to reproduce

Please add steps to reproduce the bug here along with the stack trace.

## Links to any relevant pastes or documents

Please post links to any relevant documents that might contain any extra information.

## Checklist

Use this checklist if you are using this issue as a tracker:

- [ ] Task 1
- [ ] Task 2
- [ ] Task 3 ...