Future proposal: support for test report generation

How about we add additional functionality in spread to support test report generation with task level command support to add metrics and checks results ?

So, in addition to ```MATCH``` at the task level, perhaps we can have something like:

```REPORT_CHECK_PASS ```
```REPORT_CHECK_SKIPPED ```

```REPORT_METRIC ```

My implementation currently has the report generator written in Python, and the report is posted as a comment with each PR.

It would be great to also support generating a report artifact, such as a PDF (which should be easy from the markdown/HTML/Latex used below - which is supported by GH for rendering)

My spread currently generates this:

# Test Report
Checks and metrics in this report was produced by running the tests on **x86 / noble** inside an architecture specific QEMU emulator, without KVM acceleration. Metrics related to size are accurate, but performance metrics related to time based measurements are effectively benchmarking the performance of the host machine, not the emulated platform.

## **Coverage**

Category
Check
Base
Platform
Suite
Task
Baseline
Actual




Machine boot
First boot after flashing
24.04
qemu-amd64
smoke
test-boot
Pass
Pass


qemu-arm64
Pass
Pass


qemu-riscv64
Pass
Pass


Reboot request from client
qemu-amd64
Pass
Pass


qemu-arm64
Pass
Pass


qemu-riscv64
Pass
Pass


Machine refresh
Refresh to next revision
qemu-amd64
test-refresh
Pass
Pass


qemu-arm64
Pass
Pass


qemu-riscv64
Pass
Pass

## **Metrics**
### **Machine boot to Networking/NTP**

Base
Platform
Suite
Task
Units
Baseline
Actual
Delta




24.04
qemu-amd64
smoke
test-boot
milliseconds
30000
32712
${\color{orange}\verb|9.04%|}$


qemu-arm64
30000
45793
${\color{red}\verb|52.64%|}$


qemu-riscv64
30000
28214
-5.95%

### **Machine boot to PID1**

Base
Platform
Suite
Task
Units
Baseline
Actual
Delta




24.04
qemu-amd64
smoke
test-boot
milliseconds
20000
26822
${\color{red}\verb|34.11%|}$


qemu-arm64
20000
27843
${\color{red}\verb|39.22%|}$


qemu-riscv64
20000
22214
${\color{orange}\verb|11.07%|}$

### **Machine bootloader time**

Base
Platform
Suite
Task
Units
Baseline
Actual
Delta




24.04
qemu-amd64
smoke
test-boot
milliseconds
10000
18992
${\color{red}\verb|89.92%|}$


qemu-arm64
10000
23173
${\color{red}\verb|131.73%|}$


qemu-riscv64
10000
17634
${\color{red}\verb|76.34%|}$

### **Machine reboot (only) time**

Base
Platform
Suite
Task
Units
Baseline
Actual
Delta




24.04
qemu-amd64
smoke
test-refresh
milliseconds
30000
30798
2.66%


qemu-arm64
30000
54696
${\color{red}\verb|82.32%|}$


qemu-riscv64
30000
30260
0.87%

### **Machine reboot (with refresh) time**

Base
Platform
Suite
Task
Units
Baseline
Actual
Delta




24.04
qemu-amd64
smoke
test-refresh
milliseconds
30000
38535
${\color{red}\verb|28.45%|}$


qemu-arm64
30000
54307
${\color{red}\verb|81.02%|}$


qemu-riscv64
30000
32017
${\color{orange}\verb|6.72%|}$

### **Machine shutdown time**

Base
Platform
Suite
Task
Units
Baseline
Actual
Delta




24.04
qemu-amd64
smoke
test-boot
milliseconds
3500
3044
-13.03%


qemu-arm64
3500
3187
-8.94%


qemu-riscv64
3500
3063
-12.49%

### **Disk size (compressed)**

Base
Platform
Suite
Task
Units
Baseline
Actual
Delta




24.04
qemu-amd64
smoke
test-boot
bytes
21821884
21824805
0.01%


qemu-arm64
25517168
25521977
0.02%


qemu-riscv64
34285468
34288386
0.01%

### **Disk size (decompressed)**

Base
Platform
Suite
Task
Units
Baseline
Actual
Delta




24.04
qemu-amd64
smoke
test-boot
bytes
31351390
31359612
0.03%


qemu-arm64
34602031
34605318
0.01%


qemu-riscv64
67672166
67671298
-0.00%

### **PID1 binary size**

Base
Platform
Suite
Task
Units
Baseline
Actual
Delta




24.04
qemu-amd64
smoke
test-boot
bytes
11464888
11473080
0.07%


qemu-arm64
10813624
10813624
0.00%


qemu-riscv64
10879160
10879160
0.00%

Start by reviewing spread's task-level MATCH command handling and the Python report generator described in the issue. Define how REPORT_CHECK_PASS, REPORT_CHECK_SKIPPED, and REPORT_METRIC map to the generated report, then determine the required Markdown, HTML, LaTeX, or PDF artifact behavior. Done means task results and metrics are collected and a report is posted or produced as an artifact.

Written by the indexing model from the issue text.