GTBitsOfGood / GTBitsOfGood/bog-analytics
[PMT] - Data Export CRON
- Dominant language
- TypeScript
- Stars
- 2
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
## Description
We want to build a CRON job that exports data from Bog Analytics to remote databases for teams to perform advanced queries on.
## Acceptance Criteria
- [ ] Create an ExportConfig Schema with the following attributes
```
interface ExportConfig {
_id: ObjectId;
projectId: ObjectId; // object id for project
productionDatabase: string; // encrypted databaseUrl for prod
stagingDatabase: string; // encrypted databaseUrl for staging
developmentDatabase: string; // encrypted databaseUrl for dev
}
```
- [ ] The encrypted database URLs should use a symmetric encryption scheme so we can decrypt in the CRON and encrypt when they input the database information
- [ ] Create a mongoose model for the ExportConfig Model
- [ ] Write a CRON job that exports data from bog analytics to the remote databases
- [ ] This CRON should run once a day
- [ ] You can reference the export data example project to see how it would be done: https://github.com/GTBitsOfGood/bog-analytics/tree/main/examples/export-data
- [ ] You can also see the deletion policy CRON : https://github.com/GTBitsOfGood/bog-analytics/tree/main/deletion-policy
- [ ] In the Bog Analytics Portal + Analytics API, add the ability for them to specify production, staging, and development databases + create any necessary endpoints to get / create / update an ExportConfig
## Notes
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing examples/export-data and deletion-policy to understand the existing export and CRON entry points, then trace the Bog Analytics Portal and Analytics API configuration paths. Done means an ExportConfig schema and Mongoose model, encrypted environment database URLs, daily exports to remote databases, and endpoints to get, create, and update the configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- api, backend, databases, devops
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100