rtk-ai / rtk-ai/rtk

feat(learn): extract generalizable patterns instead of raw command pairs

Open
#654 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area:cli effort-large enhancement help wanted priority:medium
Dominant language
Rust
Stars
81.1k
Forks
5.1k
Avg merge
4d 21h
Merged PRs (30d)
35

Description

Problem

rtk learn outputs raw command pairs like:

Use `aws ec2 describe-security-groups --group-ids sg-<ID> --query 'SecurityGroups[0]...' --output yaml`
not `aws ec2 authorize-security-group-egress --group-id sg-<ID> --ip-permissions IpProtocol=tcp,FromPort=8443...`

These are 200+ character commands that are too specific to be useful as reusable rules. The real lesson is: "prefer describe-* with --output table/yaml over piping JSON through inline python".

In production testing (62 sessions, 60 days), rtk learn found 44 corrections but only 1 recurred with --min-occurrences 2. The signal-to-noise ratio is too low for varied infrastructure work (Terraform, AWS CLI, kubectl, GitHub API). The feature works well for repetitive build/test/lint cycles where the same typo recurs, but most infrastructure corrections are one-off and context-dependent.

Proposed Improvement

Add pattern generalization in learn/detector.rs that extracts the principle behind a correction rather than the exact commands:

Common transformation patterns to detect
Raw correction Generalized rule
Complex jq pipeline → simpler --query JMESPath "Prefer native --query over piping through jq for AWS CLI"
Inline python3 -c "..." → native --output table "Prefer --output table/yaml over inline python formatters"
Long awk pipeline → targeted grep "Prefer simple grep over awk when only filtering lines"
find ... -name X -o -name Yfind ... \( -name X -o -name Y \) "Group -o predicates in find with parentheses"
How it could work
  1. After find_corrections() produces raw pairs, a new generalize_correction() step would:
    • Detect if the wrong command uses shell pipelines (|, inline scripts) while the right command uses native flags
    • Detect if both commands target the same base tool but differ in output formatting strategy
    • Detect common flag corrections (grouping by changed flag, not full command)
  2. deduplicate_corrections() would group by transformation type rather than exact diff token
  3. report.rs would output the generalized principle with one concrete example, not the full raw commands
Example output

Instead of the current:

Use `aws ec2 describe-security-groups --group-ids sg-<ID> --query '...' --output yaml`
not `aws ec2 describe-security-groups --group-ids sg-<ID> --query '...' --output json | python3 -c "..."`

Output:

Prefer native --output (table/yaml) over piping JSON through inline scripts (seen 3x)
  Example: aws ec2 describe-* ... --output yaml  (not ... | python3 -c "...")

Scope

  • learn/detector.rs — new generalization step after correction detection
  • learn/report.rs — output format for generalized rules
  • Significant change to the deduplication and reporting logic

This is a design proposal — happy to discuss the approach before implementing. The key question is whether generalization should happen in the detector (changing what's stored) or the reporter (changing how it's displayed).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading learn/detector.rs, learn/report.rs, and the existing correction and deduplication flow. Compare the detector-versus-reporter options described in the issue, then define how generalized rules and one concrete example should be represented and reported. Done means varied corrections produce useful grouped principles without losing recurrence counts or examples.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.