PipedreamHQ / PipedreamHQ/pipedream

[FEATURE] Support $service.db in an app file component

Open
#2,031 0 comments 0 reactions 0 assignees View on GitHub
enhancement tracked internally triaged
Dominant language
JavaScript
Stars
11.7k
Forks
5.8k
Avg merge
3d 10h
Merged PRs (30d)
102

Description

**Is your feature request related to a problem? Please describe.**

Add `$service.db` support for high level `.app.js` components.

In source development specifically, it's nice to abstract pagination, throttling & cursor tracking into the root `.app.js` component.

Then you move the implementation details downstream into the `sources/.js` file.

**Describe the solution you'd like**

Perhaps the `this.db` should be available in app components by default? Then you can just reference it?

**Do you have a workaround?**

Yes, injecting the `$service.db` in each source/action is fine. It's a little redundant but it works.

**Additional context**

When attempting to use `props` in a `.app.js`, `pd dev` gets mad:

```bash
Detected local requires, creating zip archive...
- sources/new_app_charges.js
- shopify_partner.app.js
- queries/getAppTransactions.js
error | component create api call responded with status: 400, body: {"error":"{\"name\":\"UserError\",\"message\":\"props not supported for prop shopify\"}"}
Waiting for file change (or ^C to quit)...
filechange | /Users/pierce/dev/pipedream/components/shopify_partner/shopify_partner.app.js
````

Here's the example app component:

`shopify_partner.app.js`

```javascript
require("graphql/language");
const get = require("lodash/get");
const { GraphQLClient } = require("graphql-request");

module.exports = {
type: "app",
app: "shopify_partner",
/**
* @NOTE: here is where the issue lies, you can't currently set top level props in an app module
**/
props: {
db: "$.service.db",
},
methods: {
/**
* Handle a Shopify Partner GraphQL query
*
* Includes pagination, based off of a recusion or the last stored cursor in the $service.db
*/
async query({
query,
mutation,
variables,
handleEmit,
key,
hasNextPagePath = "transactions.pageInfo.hasNextPage",
cursorPath = "transactions[0].edges[0].cursor",
}) {
const endpoint = `https://partners.shopify.com/${this.$auth.organization_id}/api/2021-04/graphql.json`;
const client = new GraphQLClient(endpoint, {
headers: {
"Content-Type": "application/json",
"X-Shopify-Access-Token": this.$auth.api_key,
},
});

// the key is unique to the source module, so we should always be getting the last message
const lastCursor = this.db.get(key);

const data = await client.request(query || mutation, {
...variables,
...(lastCursor || key ? { after: lastCursor } : {}),
});

console.log(JSON.stringify(data, null, 2));

if (data) {
handleEmit(data);
// the db becomes the source of truth for the last cursor
this.db.set(key, get(data, cursorPath));
}

// paginate the results recursively
// the hasNextPagePath references how to traverse the response from Shopify
// if hasNextPath is truthy, we continue
if (data && get(data, hasNextPagePath)) {
await this.query({
query,
mutation,
handleEmit,
variables,
});
}
},
},
};

```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.