OpenFn / OpenFn/lightning

Collections: allow me to get keys only (or support HEAD with a count header)

Open
#2,758 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Elixir
Stars
296
Forks
86
Avg merge
1d 13h
Merged PRs (30d)
50

Description

When calling out to the collections HTTP API, it would be nice to have a flag or parameter which only returns keys, rather than values.

This can be used to test query patterns and count how many items we have.

This is more useful for the CLI than workflows. But maybe the collections adaptor can implement count(pattern) and exists(pattern) APIs very cheaply with this functonality.

Related: It might also be nice to support a HEAD request to the endpoints, with a query pattern, where the number of results is written to a header. That gives us a means to count contents without downloading any data.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files or tests; start by tracing the collections HTTP API and the CLI entry points that query collections. Compare the proposed keys-only, count/exists, and HEAD approaches before choosing a scope. Done should mean an agreed way to count or retrieve keys without downloading values, with behavior covered by relevant tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
elixir
Domain
api, backend, cli
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.