GTBitsOfGood / GTBitsOfGood/bog-analytics

[PMT] - Data Export CRON

Open
#92 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
2
Forks
0
PR merge metrics
No merged PRs in 30d

Description

## Description
We want to build a CRON job that exports data from Bog Analytics to remote databases for teams to perform advanced queries on.

## Acceptance Criteria
- [ ] Create an ExportConfig Schema with the following attributes
```
interface ExportConfig {
_id: ObjectId;
projectId: ObjectId; // object id for project
productionDatabase: string; // encrypted databaseUrl for prod
stagingDatabase: string; // encrypted databaseUrl for staging
developmentDatabase: string; // encrypted databaseUrl for dev
}
```
- [ ] The encrypted database URLs should use a symmetric encryption scheme so we can decrypt in the CRON and encrypt when they input the database information
- [ ] Create a mongoose model for the ExportConfig Model
- [ ] Write a CRON job that exports data from bog analytics to the remote databases
- [ ] This CRON should run once a day
- [ ] You can reference the export data example project to see how it would be done: https://github.com/GTBitsOfGood/bog-analytics/tree/main/examples/export-data
- [ ] You can also see the deletion policy CRON : https://github.com/GTBitsOfGood/bog-analytics/tree/main/deletion-policy
- [ ] In the Bog Analytics Portal + Analytics API, add the ability for them to specify production, staging, and development databases + create any necessary endpoints to get / create / update an ExportConfig

## Notes

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing examples/export-data and deletion-policy to understand the existing export and CRON entry points, then trace the Bog Analytics Portal and Analytics API configuration paths. Done means an ExportConfig schema and Mongoose model, encrypted environment database URLs, daily exports to remote databases, and endpoints to get, create, and update the configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
api, backend, databases, devops
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.