couchbaselabs / couchbaselabs/couchbase-shell
improve support for import from files
- Dominant language
- Rust
- Stars
- 87
- Forks
- 18
- PR merge metrics
- No merged PRs in 30d
Description
Currently, there is a recipe for importing json files in a particular format. That works well, but unfortunately the format isn't always what one would like it to be. Maybe this can be mixed in with something like jq?
One thing I tried, for example, was unzipping the travel-sample database and try to reimport it. The format didn't quite match. I tried a few things listing all of the .json files and piping to reformat with builtins, but it seemed to be trying to do all of this in memory, so I killed it.
Scenario:
Given a set of files in a directory that are in JSON format, as a user I want to be able to import them. If they are not in the right format, I want to be able to massage them easily into the right format. For example, the file is likely what I really want to go into the 'content' field that kv-upsert expects, but I may want to extract a field from that to be the ID. Or, I may want the ID to be a concatination of two or more fields. Or, I may want the ID to be a sequence, possibly concatinated on a field in the file.
This may be do-able with nushell built ins, but it is complicated enough that it wasn't obvious in a few minutes.
One possible implementation: allow for a file that defines reformatting rules that are passed in to kv-upsert. Or introduce another command like kv-import.
Another solution which we could play with is jq streaming edits, which seems to be a thing, but not something I'm familiar with.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.