awslabs / awslabs/filemoverexpress
[Feature] Notifications/Transfer Event Support
- Dominant language
- TypeScript
- Stars
- 4
- Forks
- 1
- Avg merge
- 9h 54m
- Merged PRs (30d)
- 41
Description
## Summary
Add support for publishing transfer events to AWS SNS and/or SQS, allowing File Mover Express to integrate with other services and workflows when transfers complete, fail, or encounter errors.
## Background
File Mover Express already has an internal event bus (`src/cli/events/bus.go`) that fires well-defined events throughout the transfer lifecycle — job created, job complete, job error, task complete, and more. Today these events are consumed internally by the GUI and logged. They are not exposed externally.
Adding SNS/SQS as an optional event destination would allow users to:
- Trigger downstream workflows when a transfer job completes (e.g. kick off a transcoding pipeline)
- Alert on transfer failures via email, SMS, or PagerDuty through SNS subscriptions
- Feed transfer events into other services via SQS consumers
- Build audit trails of transfer activity without polling
## Existing events available to publish
The following events are already defined and carry useful data:
- `JobCompleteEvent` — job ID, name, completion time, whether any tasks errored or were skipped
- `JobErrorEvent` — job ID, name, error time, error message
- `TaskCompleteEvent` — task ID, direction (upload/download), destination path
- `JobCreateEvent` — fired when a new job is queued
- `JobProgressEvent` — periodic progress updates
## Proposed approach
Add an optional notification configuration block to the config file:
notifications:
sns:
enabled: true
topic_arn: "arn:aws:sns:us-east-1:123456789:my-topic"
events:
- job_complete
- job_error
sqs:
enabled: false
queue_url: "https://sqs.us-east-1.amazonaws.com/123456789/my-queue"
events:
- job_complete
- job_error
- task_complete
On startup, if notifications are configured, register a new listener on the existing event bus. When a matching event fires, publish it to the configured SNS topic or SQS queue as a JSON payload.
The existing AWS credentials (profile/region from the transfer profile) should be reused for SNS/SQS access.
## Implementation notes
- The event bus already supports `RegisterListener` with event type filters — a notification publisher would just be another listener
- Each event type already implements `String()` and carries structured data, making JSON serialisation straightforward
- SNS and SQS both use the AWS SDK v2 which is already a dependency
- IAM permissions for `sns:Publish` or `sqs:SendMessage` would need to be added to the setup documentation
## Relevant files
- `src/cli/events/bus.go` — event bus, `RegisterListener` is the integration point
- `src/cli/types/eventtypes/` — all event type definitions
- `src/cli/types/configtypes/config.go` — where the new notifications config block would be added
- `docs/Configuration.md` — would need updating with the new config options
- `docs/Setup.md` — would need the additional IAM permissions documented
## Acceptance criteria
- SNS and/or SQS publishing can be enabled via configuration
- Users can choose which event types trigger a notification
- Notifications are non-blocking — a failure to publish should log a warning but not affect the transfer
- Credentials use the existing AWS profile configuration
- Documentation updated for configuration and required IAM permissions
Contributor guide
Research direction
Read src/cli/events/bus.go and the event definitions under src/cli/types/eventtypes/ to understand listener registration and available payloads, then inspect src/cli/types/configtypes/config.go for existing AWS configuration. Review docs/Configuration.md and docs/Setup.md for the documentation updates. Done means configured SNS and/or SQS destinations publish selected events without blocking transfers, reuse the existing credentials, and document the required options and IAM permissions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, go
- Domain
- backend, cloud, documentation
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100