rtk-ai / rtk-ai/rtk

Monthly savings % is halved by an algebraic identity, weekly rtk/ccusage keys can never match, and `rtk session` reports adoption without checking the hook is installed

Open
#3,418 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area:cli area:docs area:security bug priority:high
Dominant language
Rust
Stars
81.1k
Forks
5.1k
Avg merge
4d 21h
Merged PRs (30d)
35

Description

Four independent correctness defects in the analytics layer. The first is provable from the arithmetic alone.

Provenance: AI-assisted source audit, human-directed, at v0.44.2 (700bdde). rtk cc-economics was never executed — it downloads ccusage via npx and reads real spend data — so rows 1, 2 and 4 are derived from source; row 3 was reproduced.

1. Monthly savings percentage is exactly half the real value

src/core/tracking.rs:104-106 documents the invariant:

/// Number of tokens saved (input - output)
pub saved_tokens: usize,
/// Savings percentage ((saved / input) * 100)
pub savings_pct: f64,

and tracking.rs:893 computes it correctly:

let savings_pct = if input > 0 { (saved as f64 / input as f64) * 100.0 } else { 0.0 };

But src/analytics/cc_economics.rs:104-108 discards that value and recomputes it with a different denominator:

self.rtk_savings_pct = Some(if stats.input_tokens + stats.output_tokens > 0 {
    stats.saved_tokens as f64
        / (stats.saved_tokens + stats.input_tokens + stats.output_tokens) as f64
        * 100.0

Substituting the documented identity saved = input − output:

saved + input + output  =  (input − output) + input + output  =  2 · input

so the reported figure is saved / (2·input) · 100exactly half of saved / input · 100. A real 60% saving is displayed as 30%.

Note also the guard tests input + output > 0 while the division uses a different sum, so the two are not even the same expression.

2. Weekly rows can never merge with ccusage

src/core/tracking.rs:800 buckets weeks with:

DATE(timestamp, 'weekday 0', '-6 days')

weekday 0 advances to the next Sunday; minus six days lands on the Monday of that week — the ISO week start, which is also what cc_economics.rs:234 declares the ccusage key to be.

But cc_economics.rs:247 then calls convert_saturday_to_monday, which adds two days on the assumption the rtk key is a Saturday. Applied to a Monday it yields Wednesday, so the rtk key and the ccusage key can never be equal and merge_weekly never joins. compute_weighted_metrics needs both sides, so every weekly row shows blank Savings and CPT.

Confirming the key side (this part I could run): on Wednesday 2026-08-05, rtk gain --weekly bucketed to 08-03 → 08-09 — Monday, not Saturday.

The function's test at :832-837 only asserts that two days are added; it never checks the input is a Saturday. Its comment also labels 2026-01-18 as a Saturday, but that date is a Sunday.

3. rtk session reports adoption without checking the hook exists

src/analytics/session_cmd.rs:42-47 counts a command as rtk-covered when classify_command says rtk could handle it — it never checks whether rtk actually ran. On a machine with no hook installed, every command still counts as covered.

Reproduced: rtk session reported 100% per session and Average adoption: 88% on a machine where rtk verify prints "RTK hook not installed".

src/analytics/gain.rs:125 already calls hook_check::status(), so the check exists in the codebase and just is not applied here.

4. Day buckets are UTC while the history list is local

tracking.rs:438 stores Utc::now() and :727/:800/:875 bucket with SQLite DATE(timestamp) — UTC. But analytics/gain.rs:260 renders the history list with .with_timezone(&Local).

So one report shows two different day boundaries. For UTC+7, commands run between local 00:00 and 07:00 are filed under the previous day. Reproduced: commands executed at local 2026-08-05 01:45 (+0700) appear under 2026-08-04. Those same UTC keys are what merge_daily joins against ccusage.

Smaller, same area
  • src/analytics/ccusage.rs:99-121 falls back to npx --yes ccusage — unpinned, no integrity check — and invokes it twice per run (a --help probe, then the real call). Fetching and executing latest-version npm code as part of a reporting command seems worth a deliberate decision rather than a fallback.
  • ccusage.rs:124 hardcodes --since 20250101 (~19 months) while the doc and the adjacent comment both say "last 90 days".
  • session_cmd.rs:16,49,174 sums output_tokens, which provider.rs:226 sets to the raw byte length, and prints it under an "Output" header — roughly 4× inflation. The sibling consumer discover/mod.rs:138-140 divides the same field by 4, as does tracking.rs.
Suggested fixes (untested — no programmer has reviewed these)
  1. Use the savings_pct already computed in MonthStats instead of recomputing, or divide by input_tokens alone.
  2. Drop convert_saturday_to_monday — the SQL already yields Monday. If some other producer really does emit Saturday keys, the conversion belongs there, guarded by a check that the input is a Saturday.
  3. Gate the adoption figure on hook_check::status() as gain.rs:125 does, or label it "coverage potential" rather than adoption.
  4. Pick one timezone for both bucketing and display — probably local for both, since the user reads days locally.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read src/analytics/cc_economics.rs, src/core/tracking.rs, src/analytics/session_cmd.rs, src/analytics/gain.rs, and src/analytics/ccusage.rs, then run the affected rtk cc-economics, rtk gain --weekly, and rtk session commands. Trace the existing tracking, hook status, timezone, and ccusage paths before changing them. Done means the four reported analytics discrepancies are corrected and the related joins and reports agree with their source data.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sqlite
Domain
analytics, cli
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.