CodeForAfrica / CodeForAfrica/HURUmap-Data

Switch to COPY instead of INSERT

Open
#7 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Some SQL files are getting large and can take a few seconds to get executed. For better performance and reducing sql file sizes, we can run the script *to update/reformat then overwrite the csv files*. The SQL will then just have to use the COPY command:

```
COPY persons(first_name,last_name,dob,email)
FROM 'C:\tmp\persons.csv' DELIMITER ',' CSV HEADER;
```

https://www.postgresqltutorial.com/import-csv-file-into-posgresql-table/

cc @kilemensi @KhadijaMahanga @ascii-dev

Contributor guide

No contributing guide indexed for this repository

Research direction

Locate the SQL files that currently contain large INSERT statements and the script or process that updates the CSV files. Compare the existing data-loading flow with PostgreSQL's COPY syntax and determine how the CSVs should be regenerated or reformatted. Done means the SQL uses COPY and the updated CSV files load successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, sql
Domain
databases
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.