awslabs / awslabs/filemoverexpress

[Feature] Notifications/Transfer Event Support

Open
#6 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
TypeScript
Stars
4
Forks
1
Avg merge
9h 54m
Merged PRs (30d)
41

Description

## Summary

Add support for publishing transfer events to AWS SNS and/or SQS, allowing File Mover Express to integrate with other services and workflows when transfers complete, fail, or encounter errors.

## Background

File Mover Express already has an internal event bus (`src/cli/events/bus.go`) that fires well-defined events throughout the transfer lifecycle — job created, job complete, job error, task complete, and more. Today these events are consumed internally by the GUI and logged. They are not exposed externally.

Adding SNS/SQS as an optional event destination would allow users to:

- Trigger downstream workflows when a transfer job completes (e.g. kick off a transcoding pipeline)
- Alert on transfer failures via email, SMS, or PagerDuty through SNS subscriptions
- Feed transfer events into other services via SQS consumers
- Build audit trails of transfer activity without polling

## Existing events available to publish

The following events are already defined and carry useful data:

- `JobCompleteEvent` — job ID, name, completion time, whether any tasks errored or were skipped
- `JobErrorEvent` — job ID, name, error time, error message
- `TaskCompleteEvent` — task ID, direction (upload/download), destination path
- `JobCreateEvent` — fired when a new job is queued
- `JobProgressEvent` — periodic progress updates

## Proposed approach

Add an optional notification configuration block to the config file:

notifications:
sns:
enabled: true
topic_arn: "arn:aws:sns:us-east-1:123456789:my-topic"
events:
- job_complete
- job_error
sqs:
enabled: false
queue_url: "https://sqs.us-east-1.amazonaws.com/123456789/my-queue"
events:
- job_complete
- job_error
- task_complete

On startup, if notifications are configured, register a new listener on the existing event bus. When a matching event fires, publish it to the configured SNS topic or SQS queue as a JSON payload.

The existing AWS credentials (profile/region from the transfer profile) should be reused for SNS/SQS access.

## Implementation notes

- The event bus already supports `RegisterListener` with event type filters — a notification publisher would just be another listener
- Each event type already implements `String()` and carries structured data, making JSON serialisation straightforward
- SNS and SQS both use the AWS SDK v2 which is already a dependency
- IAM permissions for `sns:Publish` or `sqs:SendMessage` would need to be added to the setup documentation

## Relevant files

- `src/cli/events/bus.go` — event bus, `RegisterListener` is the integration point
- `src/cli/types/eventtypes/` — all event type definitions
- `src/cli/types/configtypes/config.go` — where the new notifications config block would be added
- `docs/Configuration.md` — would need updating with the new config options
- `docs/Setup.md` — would need the additional IAM permissions documented

## Acceptance criteria

- SNS and/or SQS publishing can be enabled via configuration
- Users can choose which event types trigger a notification
- Notifications are non-blocking — a failure to publish should log a warning but not affect the transfer
- Credentials use the existing AWS profile configuration
- Documentation updated for configuration and required IAM permissions

Contributor guide

Open the contributing guide

Research direction

Read src/cli/events/bus.go and the event definitions under src/cli/types/eventtypes/ to understand listener registration and available payloads, then inspect src/cli/types/configtypes/config.go for existing AWS configuration. Review docs/Configuration.md and docs/Setup.md for the documentation updates. Done means configured SNS and/or SQS destinations publish selected events without blocking transfers, reuse the existing credentials, and document the required options and IAM permissions.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, go
Domain
backend, cloud, documentation
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.