CLI & Action: disclosure text lacks --min-confidence support (breaks documented examples and causes false positives in GitHub Actions)
- Dominant language
- Go
- Stars
- 26
- Forks
- 13
- Avg merge
- 6d 18h
- Merged PRs (30d)
- 3
Description
### Description
There is a significant CLI UX, documentation, and functional gap between `disclosure scan` and `disclosure text` regarding confidence filtering and flag naming:
1. **Documented CLI help example fails with `unknown flag: --min-confidence`:**
In [`cmd/cmd.go`](https://github.com/chaoss/disclosure/blob/main/cmd/cmd.go#L273) (lines 273 and 282), `disclosure text`'s own help text (`Long` description and `Example`) instructs users:
```sh
cat comment.txt | disclosure text --min-confidence=medium
```
However, `textCommand` does not register a `--min-confidence` flag. Running this documented command immediately fails:
```
Error: unknown flag: --min-confidence
```
2. **GitHub Action `min-confidence` is bypassed for PR bodies, causing false positives:**
In [`action/action.yml`](https://github.com/chaoss/disclosure/blob/main/action/action.yml#L18-L23), the `min-confidence` input documentation states:
```yaml
# set min-confidence to medium or above if your PR template has AI tools mentioned in comments etc.
# this will help avoid false positives especially when using AI use checkboxes.
```
However, because `disclosure text` lacks `--min-confidence` support, [`action/action.yml`](https://github.com/chaoss/disclosure/blob/main/action/action.yml#L97-L103) invokes `ai-detection text` without `--min-confidence`:
```bash
TEXT_REPORT=$(echo "${PR_BODY}" | ${{ runner.temp }}/ai-detection text \
--format=json ${CHECKBOX_DETECTION_FLAG} \
--cb-disclosed-ai="${CHECKBOX_LABEL_AI_USED}" \
--cb-disclosed-noai="${CHECKBOX_LABEL_AI_NOT_USED}") || TEXT_EXIT=$?
if [ "${TEXT_EXIT:-0}" = "1" ]; then
AI_DETECTED=true
fi
```
Whenever a PR body mentions any tool name (e.g., in PR template instructions, comments, or disclosure checkboxes), `toolmention` generates low-confidence findings (score 20). Because `textCommand` does not filter by confidence, any finding causes an exit code of `1` (`ExitAI`).
Consequently, the action sets `ai-detected=true` and flags/labels the PR as AI-detected, completely bypassing the user-configured `min-confidence: 'medium'` or `'high'` threshold.
3. **Documented CLI usage flags in `README.md` do not exist:**
In [`README.md`](https://github.com/chaoss/disclosure/blob/main/README.md#cli-usage) (lines 30-35):
```
disclosure text \
[--format=json|text] [--input=FILE|-] \
[--checkbox-label-ai-used="AI was used"] \
[--checkbox-label-ai-not-used="AI was not used"]
```
- Flags `--checkbox-label-ai-used` and `--checkbox-label-ai-not-used` do not exist in the CLI (renamed in PR #81 commit `45a48c0` to `--cb-disclosed-ai` and `--cb-disclosed-noai`). Passing them fails with `unknown flag: --checkbox-label-ai-used`.
- The `--enable-checkbox-detection` flag is omitted from the synopsis.
- In `cmd/cmd.go` line 289, the example snippet is missing a line continuation backslash `\` after `--enable-checkbox-detection`.
---
### Steps to Reproduce
1. Execute the command documented in `disclosure text --help`:
```sh
echo "Claude was mentioned" | disclosure text --min-confidence=medium
```
**Output:**
```
Error: unknown flag: --min-confidence
```
2. Configure the GitHub Action with `min-confidence: 'medium'` and `scan-pr-body: 'true'`:
- Open a PR containing a tool mention in its description (e.g. `Claude` in template instructions).
- Although the commit scan filters by `medium`, `disclosure text` outputs the low-confidence finding (score 20) and exits with `1`.
- The action sets `ai-detected=true` despite `min-confidence: 'medium'`.
---
### Expected Behavior
1. `disclosure text` should support `--min-confidence` (and `--confidence-levels`), filtering findings below the specified threshold. If all findings are below `min-confidence`, it should exit with `0` (`ExitNoAI`).
2. `action/action.yml` should pass `--min-confidence="${MIN_CONFIDENCE}"` to the text scan command so that PR body scans respect the configured confidence threshold.
3. Help text and `README.md` should accurately document the active flags (`--cb-disclosed-ai`, `--cb-disclosed-noai`, `--enable-checkbox-detection`, and `--min-confidence`).
---
### Proposed Solution
1. **`cmd/cmd.go`**:
- Add `--min-confidence` (and optionally `--confidence-levels`) flags to `textCommand`.
- Filter `findings` based on `minConf` before formatting output and setting `exitCode`.
- Add missing trailing backslash `\` in `cmd/cmd.go` line 289.
2. **`action/action.yml`**:
- Pass `--min-confidence="${MIN_CONFIDENCE}"` to the `ai-detection text` invocation.
3. **`README.md`**:
- Update `disclosure text` usage synopsis to reflect `--cb-disclosed-ai`, `--cb-disclosed-noai`, `--enable-checkbox-detection`, and `--min-confidence`.
Contributor guide
Research direction
Start in cmd/cmd.go at textCommand and compare its flags, help text, and exit-code handling with the scan command. Then inspect the text invocation in action/action.yml and the CLI synopsis in README.md. Done means documented flags work, PR-body scans pass the configured confidence threshold, and the help and README examples match the active flag names.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, go
- Domain
- ci-cd, cli, documentation
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100