PipedreamHQ / PipedreamHQ/pipedream

[Refactor / Improvement] Fetch historical events for Webhook-Based Event Sources

Open
#4,652 2 comments 0 reactions 0 assignees View on GitHub
enhancement triaged trigger / source
Dominant language
JavaScript
Stars
11.7k
Forks
5.8k
Avg merge
3d 10h
Merged PRs (30d)
102

Description

[Resurrecting a task we've done before:](https://github.com/PipedreamHQ/pipedream/issues/3332)

**Problem:**
Many of our webhook-based sources (Telegram Bot for example), require users to manually trigger an eligible event to get picked up by the trigger, in order for a test event to show appear when they're building a new workflow. This is not always obvious to the user, so when they can't figure out how to do this, sometimes they give up.

**Solution:**
We should do this automatically for users whenever possible, by fetching historical data from the relevant GET endpoint on the deploy() hook, then on activate() we create the webhook like normal (Andrew did this for [Gorgias here](https://github.com/PipedreamHQ/pipedream/blob/67d30c91c1ee7615ea0a85cd579918e3400dfe21/components/gorgias/sources/common/base.mjs), and we already do this for Google Sheets, Slack, and a couple other apps). We should fetch the most recent 25 events and order them in descending order of recency.

**Note:** The shape of the event can be different for the event that arrives via webhook vs. the event retrieved via the REST API. Ideally we should format the events returned from the REST API in the same way as they would normally arrive via webhook, so that the shape of the event emitted from the source is the same in both cases. If this is overly difficult or not possible, please flag that in the appropriate ticket.

**Apps to implement this strategy in (in rough order of priority):**
- [ ] Salesforce
- [ ] Typeform
... more to come

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.