HarperFast / HarperFast/harper
The table-size analytics metric is never emitted on RocksDB
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
## Summary
`METRIC.TABLE_SIZE` (`table-size`) has exactly one emitter, and it sits on the **non-RocksDB** branch of `storeDBSizeMetrics`. Since RocksDB is the default (and LMDB is deprecated), the metric is absent on effectively every current deployment — while being documented without qualification in the analytics reference.
## Mechanism
`resources/analytics/write.ts`:
```ts
if (firstTable.primaryStore instanceof RocksDatabase) {
// ... database-size from SST totals
storeMetric(analyticsTable, metric);
} else {
const dbUsedSize = storeTableSizeMetrics(analyticsTable, db, tables); // <-- only caller
// ... database-size with used/free/audit
}
```
`storeTableSizeMetrics` is the only place that writes `METRIC.TABLE_SIZE`, and line 391 is its only call site. So on RocksDB:
- no `table-size` rows are written at all
- `database-size` is written without `used` / `free` / `audit`
The per-table RocksDB path (`storeRocksDBStatsMetrics`) does emit a per-table row every cycle, but as `rocksdb-stats`, carrying only `numRunningCompactions` / `compactionPending` — no size.
`Table.getSize()` already handles both engines (`rocksdb.estimate-live-data-size` for RocksDB), so the value is available; nothing consumes it on this path.
## Impact
- Any dashboard or alert built on `table-size` is silently empty on a RocksDB install. Absence reads as "no data yet", not "this metric does not exist here".
- [Analytics reference — Resource Usage Metrics](https://docs.harperdb.io/docs/reference/analytics/overview) lists `table-size` with `database`, `table` dimensions and no engine qualifier.
- It also blocks putting anything else per-table on that record. Found while adding record-structure dictionary counts for #2220 — the fields would have been written into a row that is never emitted, so that hunk was dropped.
## Open question before fixing
Emitting a `table-size` row per table per aggregation cycle (60s default) is real cardinality — ~525k rows/table/year in `hdb_analytics`. Worth deciding whether the RocksDB emission should be:
1. per aggregation cycle, matching the LMDB branch, or
2. gated on `analytics.storageInterval` (default every 10th cycle), matching how `storage-volume` is sampled,
before wiring it. Option 2 seems better proportioned to how slowly the value moves; the docs should state whichever is chosen.
Alternatively, if `table-size` is considered obsolete on RocksDB, it should be removed from the reference rather than left documented and dead.
Contributor guide
Research direction
Read resources/analytics/write.ts at storeDBSizeMetrics, comparing the RocksDB and non-RocksDB branches, then inspect Table.getSize(). Decide whether table-size should be emitted per cycle, sampled, or removed; done means the chosen behavior is implemented and the analytics reference matches it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- nodejs, typescript
- Domain
- analytics, databases
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100