adobe / adobe/elixir-dumper

Use approximate row counts where possible

Open
#7 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Elixir
Stars
36
Forks
3
PR merge metrics
No merged PRs in 30d

Description

Dumper works great for small tables but can quickly get into timeouts when viewing very large tables. This is largely due to the COUNT() query requiring a sequential scan of the table.

In PostgreSQL it's possible to get approximate row counts using:

```
SELECT reltuples::BIGINT AS estimate
FROM pg_class
WHERE oid = 'your_table_name'::regclass;
```

In MyXQL it's:

```
SELECT table_rows
FROM information_schema.tables
WHERE table_schema = 'your_database_name'
AND table_name = 'your_table_name';
```

Both are approximate and get update when ANALYZE / VACUUM are run.

I find little value in having the exact number of rows in a view such as:

![Image](https://github.com/user-attachments/assets/5f1831dd-0a6d-425d-937a-c8385fa49164)

I think Dumper should leverage approximate row counts where possible and simply display this as "Showing at most [x] out of ~1289317".

Contributor guide

Open the contributing guide

Research direction

Start by locating the Dumper code that issues the exact COUNT() query and inspect how PostgreSQL and MyXQL database access are separated. Determine how approximate row counts can be retrieved for both databases and how the view formats its current count. Done means large-table views use approximate counts where supported and display the requested approximate wording without the timeout-prone scan.

Written by the indexing model from the issue text.

Assessment

Tech stack
elixir, mysql, postgresql
Domain
database
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.