[Feature Request] : Is it possible for apache beam mongo sdk to include support to read Mongodb cdc (Change data capture) streams ?
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
### What would you like to happen?
Hi folks , There are many use cases where people want realtime migration of collections of Mongodb to Big query (Data Warehouse ) / GCS (Data lake). Hence , it would be great if apache beam mongo sdk included support to read Mongodb cdc (Change data capture) streams in real time and those events capture can be dumped into big query or GCS for analytical purposes. Currently to achieve this stuff we need a lot complex architecture in GCP including debezium , kafka etc. If this feature is implemented properly there will be only one single pipeline which will do all the stuff.
### Issue Priority
Priority: 3
### Issue Component
Component: io-py-mongodb
Contributor guide
Assessment
This issue has not been assessed yet.