anthropics / anthropics/financial-services
extract_numbers.py: a CORRECT multi-year deck reports 5 'high severity' inconsistencies — grouping ignores the period
- 主要语言
- Python
- 星标
- 34.8k
- 派生
- 5.2k
- 平均合并
- 2 小时 2 分钟
- 30 天内合并 PR
- 3
描述
## Summary
`find_inconsistencies` in `skills/ib-check-deck/scripts/extract_numbers.py`
groups figures by **category alone** and flags anything more than 5% from the
largest group. A deck carrying FY2023, FY2024 and FY2025E revenue is therefore
self-contradictory by construction — which is every deck with a financial
history.
Measured on a four-line deck: **a correct deck produced 5 "high severity"
inconsistencies**, against **1** for a deck containing a genuine contradiction.
Precision ≈ 1/6. The true finding is indistinguishable from the noise.
## Reproduce
`clean.md` — correct, no contradiction:
```markdown
## Slide 2 — Historical Performance
Revenue grew from $100.0 million in FY2023 to $120.0 million in FY2024.
EBITDA was $20.0 million in FY2023 and $25.0 million in FY2024.
## Slide 5 — Projections
Revenue of $145.0 million is forecast for FY2025E.
```
`planted.md` — one genuine contradiction (FY2024 revenue stated twice):
```markdown
## Slide 2 — Historical Performance
Revenue was $120.0 million in FY2024.
## Slide 7 — Transaction Summary
FY2024 revenue of $135.0 million supports the valuation.
```
```
$ python extract_numbers.py clean.md --check -> 5 inconsistencies (all severity "high")
$ python extract_numbers.py planted.md --check -> 1 inconsistency
```
On the clean deck it reports `revenue $100.0million vs $120.0million`,
`vs $145.0million`, and `ebitda $20.0million vs $25.0million` — all correct
figures for different periods. (One of the five is `2023t`, the separate
parsing defect filed alongside this.)
## Cause
```python
# lines ~192–210
for num in numbers:
if num.category != 'other':
by_category[num.category].append(num) # period is discarded
...
if diff_pct < 0.05: # 5% tolerance on VALUE only
```
Nothing distinguishes "the same metric stated twice inconsistently" from "the
same metric across three periods".
## Suggested fix
Key the comparison on **(category, period)**, not category alone:
- parse a period marker near each figure — `FY2024`, `FY24`, `2025E`, `2023A`;
- attribute a figure to the period that **follows** it before the next figure,
falling back to the preceding one. Nearest-by-distance is wrong on the
commonest sentence there is: in *"EBITDA was $20.0m in FY2023 and $25.0m in
FY2024"*, `25.0` sits 13 chars after `FY2023` and 18 before `FY2024`, so
distance alone assigns both figures to FY2023 and re-creates the false
positive;
- only report when two values share a category **and** a period;
- skip figures with no resolvable period rather than pooling them.
With that change the two fixtures above give **0** and **1** respectively.
Happy to open a PR.
贡献指南
这个仓库没有索引到贡献指南
评估
这个 Issue 还没有评估数据。