elastic / elastic/ai-github-actions

[autonomy-atomicity] safe-input extraction helper relies on YAML key order across PR readiness tests

Open Beginner friendly
#1,771 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
11
Forks
16
Avg merge
22h 9m
Merged PRs (30d)
31

Description

## Autonomy / Atomicity Findings

### 1. Test helper implicitly binds to first `safe-inputs` entry instead of a named key
**Category:** Ordering dependency
**File(s):** `tests/test_safe_input_ready_to_make_pr.py` (lines 28-40, 141-143, 171-173, 258-260, 326-328, 376-378, 442-444, 525-527)
**Problem:**
The shared helper `extract_py_block()` reads YAML frontmatter and selects the script with:

```python
first_key = next(iter(safe_inputs))
py_code = safe_inputs[first_key]["py"]
```

Most test classes then reuse this helper via fixtures that load `PUSH_FRAGMENT` or `CREATE_FRAGMENT`. This creates an implicit ordering dependency: if any contributor adds another `safe-inputs` entry (or reorders keys) in either fragment, many unrelated tests can suddenly execute the wrong script and fail. In parallel development, two valid changes that touch frontmatter structure and test coverage can collide through this hidden key-order coupling.

**Suggested fix:**
Make extraction explicit and deterministic by passing the expected safe-input key into the helper (for example, `ready-to-push-to-pr` or `ready-to-make-pr`) and fail fast if that key is missing. This removes ordering dependence and isolates tests from unrelated frontmatter growth.

## Suggested Actions
- [ ] Refactor `extract_py_block()` to require an explicit `safe_input_key` argument.
- [ ] Update push/create fixtures to pass the exact expected key for each fragment.
- [ ] Add one guard test that a fragment with multiple `safe-inputs` entries still extracts the intended key deterministically.

---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Trigger Autonomy Atomicity Analyzer](https://github.com/elastic/ai-github-actions/actions/runs/31026051549)

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

Contributor guide

Open the contributing guide

Research direction

Start in tests/test_safe_input_ready_to_make_pr.py by reading extract_py_block() and the PUSH_FRAGMENT and CREATE_FRAGMENT fixtures at the referenced lines. Require the intended safe-input key, update each fixture to pass it, and add the requested guard test; done means multiple safe-inputs entries still select the named script deterministically.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Refactor
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.