Support commenting on a PR when trigger is "issue_comment" - Action is not creating any comments on PR

I was trying to setup a Github action that runs when I open a PR and add a comment with the content `/benchmark`.
Here's my workflow file:
```yaml
name: Benchmark test

on:
issue_comment:
types: [created, edited, deleted]

jobs:
run-benchmark:
if: contains(github.event.comment.html_url, '/pull/') && contains(github.event.comment.body, '/benchmark')
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run benchmark
run: |
python3 gen_random_result.py | tee ${GITHUB_WORKSPACE}/output.json
- name: Download previous benchmark data
uses: actions/cache@v4
with:
path: ./cache
key: ${{ runner.os }}-benchmark

- name: Store new result
uses: benchmark-action/github-action-benchmark@v1
with:
tool: customSmallerIsBetter
output-file-path: output.json
external-data-json-path: ./cache/benchmark-data.json
github-token: ${{ secrets.GITHUB_TOKEN }}
summary-always: true
comment-always: true
```

The `gen_random_result.py` script does exactly what its name says, it just generates some random data of the form
```json
[
{
"name": "Test benchmark",
"unit": "Seconds",
"value": "0.4937825918725717"
}
]
```

The action runs as expected but it doesn't generate a comment on the PR with the results.
Is there something I'm missing?