OpenFn / OpenFn/adaptors

Collections: should get return an object or an array?

Open
#863 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

DevX Needs Priority
Dominant language
JavaScript
Stars
24
Forks
41
Avg merge
3d 13h
Merged PRs (30d)
12

Description

When I do collections.get('collection', '*'), the returned state.data is an array of {key, value} pairs. The array has a cursor key.

Maybe I've just confused myself after doing the CLI, but shouldn't we return an object of keys and values? Ie, { key1: { ... },key2: { ...} }

We'd probably have to nest it in an { items, cursor } object to expose the cursor to the user.

Some considerations:

  • The CLI will output and object of key/value pairs. Surely the adaptor and CLI should use the same structure?
  • each will call the callback with state, value, key. No object.
  • If I return an object, that's a bit harder to iterate over. I can't do state.data.map(({value}) => { ... }). I probably can't pass it into common.each.
  • the {key, value } structure allows us to return metadata, like created and udpated timestamps, ie {key, value, created, updated }. This was actually the initial design and is what the wire format should look like. If we don't use this structure, how do we handle metadata? If we return a flat object there's no space. We'd have to add a non-iteratable meta key or something

Hmm. Maybe we're doing the right thing with the array. I'll think about this overnight.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names collections.get, state.data, each, common.each, and the CLI as the relevant entry points. Start by tracing the current collection response and CLI representation, then compare iteration and metadata requirements. Done requires an agreed return shape and a documented or tested behavior for cursor and metadata.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.