raystack / raystack/meteor

Add event receiver framework for webhook-driven graph updates

Open
#508 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
242
Forks
45
PR merge metrics
No merged PRs in 30d

Description

Context

Not every metadata change needs a full extraction cycle. Many sources can push notifications — a GitHub webhook on a schema migration merge, a PagerDuty alert when an incident opens, a cloud event when a resource is created. Meteor should be able to receive these events and convert them into graph updates.

Scope

  • Lightweight HTTP server for receiving webhooks (meteor receive or built into meteor run)
  • Pluggable event handler interface — map incoming events to asset/relationship mutations
  • Built-in handlers for common sources:
    • GitHub webhooks (push, PR merge, release)
    • Generic cloud events (GCP Pub/Sub, AWS EventBridge)
  • Feed received events into the same processing pipeline as extracted records
  • Authentication and validation for incoming webhooks

Design Considerations

  • Event receivers complement extractors, they don't replace them. Full extraction provides the baseline; events provide real-time updates.
  • Should support running alongside batch extraction in the same Meteor process
  • Events should carry enough metadata to route to the right graph update

Why

Batch extraction on a 1-hour schedule means the graph can be up to 1 hour stale. Event receivers close this gap for sources that support push notifications, making the graph near-real-time for critical changes.

References

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with docs/docs/roadmap.md and the existing meteor run and extraction pipeline; the issue does not identify implementation files or tests. The work is complete when webhook receiving, pluggable handlers, built-in source support, validation, and graph-pipeline integration are defined and covered by tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.