simonw / simonw/sqlite-utils

Special handling for SQLite column of type `JSON`

Open
#579 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.2k
Forks
172
Avg merge
9m
Merged PRs (30d)
1

Description

sqlite-utils should detect and have specially handling for column with a JSON column. For example:

CREATE TABLE "dogs" (
  id INTEGER PRIMARY KEY,
  name TEXT,
  friends JSON 
);

Automatic Nesting

According to "Nested JSON Values", sqlite-utils will only expand JSON if the --json-cols flag is passed. It looks like it'll try to json.load all text column to test if its JSON, which can get expensive on non-json columns.

Instead, sqlite-utils should be default (ie without the --json-cols flags) do the maybe_json() operation on columns with a declared JSON type. So the above table would expand the "friends" column as expected, withoutthe --json-cols flag:

sqlite-utils dogs.db "select * from dogs" | python -mjson.tool
[
    {
        "id": 1,
        "name": "Cleo",
        "friends": [
            {
                "name": "Pancakes"
            },
            {
                "name": "Bailey"
            }
        ]
    }
]

I'm sure there's other ways sqlite-utils can specially handle JSON columns, so keeping this open while I think of more

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the sqlite-utils dogs.db "select * from dogs" CLI path and trace how --json-cols invokes maybe_json() for selected columns. Check how declared SQLite column types are exposed, then add coverage for automatic JSON expansion on a column declared JSON without the flag; the example query should produce nested friends values.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, sqlite
Domain
cli, database
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.