blaze / blaze/odo

gzip csv files don't work with our custom sql generation

Open
#289 0 comments 1 reaction 1 assignee Claimed by @cpcloud View on GitHub
bug enhancement
Dominant language
Python
Stars
1k
Forks
131
PR merge metrics
No merged PRs in 30d

Description

there are a few options here:
- error out before the database tries to read anything if there isn't a way to do this natively
- error out on everything except postgres + UNIX OSs (postgres has a `PROGRAM` clause in its `COPY` command that allows use of an external program to e.g., gzcat the file in)
- do the above for postgres, and for everything else enforce this path `csv -> chunks(temp(csv)) -> sa.Table`
- to enforce this without adding a bunch of conditionals, it might make sense to have a `GZip` metatype

cc @llllllllll @ltransom

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.