DataTalksClub / DataTalksClub/datamailer
Process inbound SES mail and publish normalized email events
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Context
The Zapier replacement needs two inbound-email routes: TODO email and invoice/receipt email. Datamailer already has inbound MIME inspection code and test fixtures, while production infrastructure already has parts of an SES-to-S3 mailbox path. Datamailer should own receipt, parsing, alias routing, and normalization, then publish events for Dapier.
Acceptance criteria
- Integrate SES receipt rules, encrypted private S3 storage, S3 event/SQS processing, a DLQ, and least-privilege worker permissions.
- Route configured recipient aliases such as invoice and TODO independently.
- Parse sender, recipients, subject, date, Message-ID, text, HTML, and attachments.
- Reuse/refactor the existing MIME parser and fixtures rather than duplicating parsing.
- Store raw/large content in S3 and publish references rather than embedding binaries.
- Use Message-ID plus recipient route for idempotency.
- Publish a documented, versioned normalized event for Dapier.
- Document raw MIME retention and deletion.
- Add tests for HTML/text alternatives, attachments, multiple recipients, duplicate delivery, and malformed MIME.
- Add a production smoke test for a real received message.
- Update the corresponding DataTalksClub/aws-infra Datamailer root and document that cross-repository change.
Boundary
Datamailer does not render PDFs or decide DataOps invoice/TODO behavior. Cross-repository plan: dapier/docs/zapier-migration-plan.md.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating Datamailer’s existing MIME parser and test fixtures, then read dapier/docs/zapier-migration-plan.md and the corresponding DataTalksClub/aws-infra Datamailer root. Map the SES, S3, SQS, DLQ, alias-routing, and event-publishing boundaries before changing them. Done means the listed parsing, idempotency, storage, event, documentation, cross-repository, and production smoke-test criteria are covered.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, django, python
- Domain
- backend, cloud, devops, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100