bcgov / bcgov/entity

DRS - Create SFTP Service to Receive BCMail+ Scanned Documents

Open
#34,626 0 comments 0 reactions 1 assignee Claimed by @doug-lovett View on GitHub
Colin Egress
Dominant language
JavaScript
Stars
23
Forks
62
Avg merge
24m
Merged PRs (30d)
1

Description

All pre-registries scanning (pre 2004?) paper documents are now being scanned by BCMail+. The final destination of the digital versions is GCP cloud storage and the DRS. Create a new SFTP service to receive these documents and create associated DRS document records.

Email John to get BCMail+ file transfer capabilities:
Can they transfer files to us via SFTP with public key authentication (Push to us instead of us pulling from them)?
Do they support the GCP Cloud Storage API to upload documents directly to GCP buckets?
Can they call an HTTPS endpoint to POST a document?

SFTP service options

- [x] Option 1
Try out GCP Cloud FTP https://docs.cloud.google.com/cloud-ftp/overview in DEV as the preferred option. Unfortunately Canada is not currently in the set of supported locations for running this service (https://docs.cloud.google.com/cloud-ftp/locations). It is possible to get an exemption if the EU or US hosts this service. The uploaded documents will be stored in Canada but the data will go through a service hosted in another country.
- Create an external SFTP server
- Create a test user
- Create a test user bucket and service account
- Test with a BCMail+ file.

- [ ] Option 2
Use a custom solution similar to the SFTPGo option here: https://github.com/DinoChiesa/sftp-server-in-gce/tree/main?tab=readme-ov-file

- [x] Update the DRS API to receive notifications when a new BCMail+ file is transferred
- Add a new callback endpoint.
- Create a new DRS record.
- Move the BCMail+ file to the permanent cloud storage location

Basic information with the entity type and entity id is in the BCMail+ filename. For example, "BC 0204750.pdf".

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.