couchbaselabs / couchbaselabs/couchbase-shell

improve support for import from files

Open
#16 3 comments 0 reactions 1 assignee Claimed by @daschl View on GitHub
Dominant language
Rust
Stars
87
Forks
18
PR merge metrics
No merged PRs in 30d

Description

Currently, there is a recipe for importing json files in a particular format. That works well, but unfortunately the format isn't always what one would like it to be. Maybe this can be mixed in with something like jq?

One thing I tried, for example, was unzipping the travel-sample database and try to reimport it. The format didn't quite match. I tried a few things listing all of the .json files and piping to reformat with builtins, but it seemed to be trying to do all of this in memory, so I killed it.

Scenario:
Given a set of files in a directory that are in JSON format, as a user I want to be able to import them. If they are not in the right format, I want to be able to massage them easily into the right format. For example, the file is likely what I really want to go into the 'content' field that kv-upsert expects, but I may want to extract a field from that to be the ID. Or, I may want the ID to be a concatination of two or more fields. Or, I may want the ID to be a sequence, possibly concatinated on a field in the file.

This may be do-able with nushell built ins, but it is complicated enough that it wasn't obvious in a few minutes.

One possible implementation: allow for a file that defines reformatting rules that are passed in to kv-upsert. Or introduce another command like kv-import.

Another solution which we could play with is jq streaming edits, which seems to be a thing, but not something I'm familiar with.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.