PipedreamHQ / PipedreamHQ/pipedream
[FEATURE] Support $service.db in an app file component
- Dominant language
- JavaScript
- Stars
- 11.7k
- Forks
- 5.8k
- Avg merge
- 3d 10h
- Merged PRs (30d)
- 102
Description
**Is your feature request related to a problem? Please describe.**
Add `$service.db` support for high level `.app.js` components.
In source development specifically, it's nice to abstract pagination, throttling & cursor tracking into the root `.app.js` component.
Then you move the implementation details downstream into the `sources/.js` file.
**Describe the solution you'd like**
Perhaps the `this.db` should be available in app components by default? Then you can just reference it?
**Do you have a workaround?**
Yes, injecting the `$service.db` in each source/action is fine. It's a little redundant but it works.
**Additional context**
When attempting to use `props` in a `.app.js`, `pd dev` gets mad:
```bash
Detected local requires, creating zip archive...
- sources/new_app_charges.js
- shopify_partner.app.js
- queries/getAppTransactions.js
error | component create api call responded with status: 400, body: {"error":"{\"name\":\"UserError\",\"message\":\"props not supported for prop shopify\"}"}
Waiting for file change (or ^C to quit)...
filechange | /Users/pierce/dev/pipedream/components/shopify_partner/shopify_partner.app.js
````
Here's the example app component:
`shopify_partner.app.js`
```javascript
require("graphql/language");
const get = require("lodash/get");
const { GraphQLClient } = require("graphql-request");
module.exports = {
type: "app",
app: "shopify_partner",
/**
* @NOTE: here is where the issue lies, you can't currently set top level props in an app module
**/
props: {
db: "$.service.db",
},
methods: {
/**
* Handle a Shopify Partner GraphQL query
*
* Includes pagination, based off of a recusion or the last stored cursor in the $service.db
*/
async query({
query,
mutation,
variables,
handleEmit,
key,
hasNextPagePath = "transactions.pageInfo.hasNextPage",
cursorPath = "transactions[0].edges[0].cursor",
}) {
const endpoint = `https://partners.shopify.com/${this.$auth.organization_id}/api/2021-04/graphql.json`;
const client = new GraphQLClient(endpoint, {
headers: {
"Content-Type": "application/json",
"X-Shopify-Access-Token": this.$auth.api_key,
},
});
// the key is unique to the source module, so we should always be getting the last message
const lastCursor = this.db.get(key);
const data = await client.request(query || mutation, {
...variables,
...(lastCursor || key ? { after: lastCursor } : {}),
});
console.log(JSON.stringify(data, null, 2));
if (data) {
handleEmit(data);
// the db becomes the source of truth for the last cursor
this.db.set(key, get(data, cursorPath));
}
// paginate the results recursively
// the hasNextPagePath references how to traverse the response from Shopify
// if hasNextPath is truthy, we continue
if (data && get(data, hasNextPagePath)) {
await this.query({
query,
mutation,
handleEmit,
variables,
});
}
},
},
};
```
Contributor guide
Assessment
This issue has not been assessed yet.