blaze / blaze/odo

How do I write a JSON column to a postgres table?

Open
#572 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1k
Forks
131
PR merge metrics
No merged PRs in 30d

Description

I'm trying to write a pandas dataframe that contains a JSON column into a postgres database using odo. Here is what I have:

```from odo import odo odo(df, 'postgres://localhost:1234/testdb::test_table')```

test_table has a JSON column called data. odo is unable to copy the dataframe and gives me 

```*** NotImplementedError: No SQL-datashape match for type JSON.```

I then tried to explicitly specify the JSON type for the column:

```
odo(df, 'postgres://localhost:1234/testdb::test_table',
dshape='var * {data: json}')
*** TypeError: DataShape json is not NumPy-compatible
```

As you can see, it then gives me an error that DataShape json is not Numpy-compatible. How do I go about doing this with odo? Pandas to_sql works without a hitch, btw. The only problem is it's really slow.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the odo(df, 'postgres://localhost:1234/testdb::test_table') example with a pandas dataframe containing a JSON column. Trace the SQL-datashape matching and NumPy compatibility errors, then verify that writing the dataframe to the PostgreSQL table succeeds while preserving the JSON data.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, postgresql, python
Domain
data, databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.