turicas / turicas/rows

csv_to_sqlite should convert values according to schema

Open
#307 0 comments 0 reactions 1 assignee View on GitHub

@turicas is already working on this.

Since Feb 15, 2019.

bug
Dominant language
Python
Stars
886
Forks
137
PR merge metrics
No merged PRs in 30d

Description

If the schema is detected or specified, the rows.utils.csv_to_sqlite function must use the field classes to convert the values before exporting to SQL.

To reproduce the bug, create two files:

echo -e "value\n123\n   \n456" > data.csv
echo -e "field_name,field_type\nvalue,decimal" > test-schema.csv

Then, call the function via CLI:

rows csv2sqlite --schemas=test-schema.csv data.csv data.sqlite
# Prints: "ValueError: could not convert string to float: "

Note: doing this conversion always will slow down the process, but is the more robust way of converting if a schema file is specified (not the case when schema is detect). May add a --convert-from-schema parameter to run this conversion.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.