electric-sql / electric-sql/pglite
Initial Bulk Loading Big Database With Indexing
- Dominant language
- TypeScript
- Stars
- 16k
- Forks
- 442
- Avg merge
- 20h 19m
- Merged PRs (30d)
- 7
Description
Hi!
I am relying on the library to load some data in a PWA app which bulk loads large amount of data using the `COPY` command (with `.csv` files). I have a specific table i.e `big_table` which has roughly 120k rows and I have a `pg_trgm` index on it for fuzzy matching on the client side.
```sql
CREATE INDEX IF NOT EXISTS trgm_idx_slide_big_table
ON big_table
USING GIST ((
coalesce(remove_diacritics(content), '') || ' ' ||
coalesce(remove_diacritics(header), '')) gist_trgm_ops );
```
The current problem is that indexing alone takes around the same amount of time of bulk importing the data. The data import takes around ~10s and the Indexing takes around ~14s.
The whole database is around ~130 MB. Is there any other efficient way that I could use for faster insertion initially?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.