elastic / elastic/ai-github-actions
[autonomy-atomicity] safe-input extraction helper relies on YAML key order across PR readiness tests
- Dominant language
- Python
- Stars
- 11
- Forks
- 16
- Avg merge
- 22h 9m
- Merged PRs (30d)
- 31
Description
## Autonomy / Atomicity Findings
### 1. Test helper implicitly binds to first `safe-inputs` entry instead of a named key
**Category:** Ordering dependency
**File(s):** `tests/test_safe_input_ready_to_make_pr.py` (lines 28-40, 141-143, 171-173, 258-260, 326-328, 376-378, 442-444, 525-527)
**Problem:**
The shared helper `extract_py_block()` reads YAML frontmatter and selects the script with:
```python
first_key = next(iter(safe_inputs))
py_code = safe_inputs[first_key]["py"]
```
Most test classes then reuse this helper via fixtures that load `PUSH_FRAGMENT` or `CREATE_FRAGMENT`. This creates an implicit ordering dependency: if any contributor adds another `safe-inputs` entry (or reorders keys) in either fragment, many unrelated tests can suddenly execute the wrong script and fail. In parallel development, two valid changes that touch frontmatter structure and test coverage can collide through this hidden key-order coupling.
**Suggested fix:**
Make extraction explicit and deterministic by passing the expected safe-input key into the helper (for example, `ready-to-push-to-pr` or `ready-to-make-pr`) and fail fast if that key is missing. This removes ordering dependence and isolates tests from unrelated frontmatter growth.
## Suggested Actions
- [ ] Refactor `extract_py_block()` to require an explicit `safe_input_key` argument.
- [ ] Update push/create fixtures to pass the exact expected key for each fragment.
- [ ] Add one guard test that a fragment with multiple `safe-inputs` entries still extracts the intended key deterministically.
---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Trigger Autonomy Atomicity Analyzer](https://github.com/elastic/ai-github-actions/actions/runs/31026051549)
Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
Contributor guide
Research direction
Start in tests/test_safe_input_ready_to_make_pr.py by reading extract_py_block() and the PUSH_FRAGMENT and CREATE_FRAGMENT fixtures at the referenced lines. Require the intended safe-input key, update each fixture to pass it, and add the requested guard test; done means multiple safe-inputs entries still select the named script deterministically.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing
- Issue type
- Refactor
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 76/100