denodrivers / denodrivers/postgres
[feature] Support streaming of buffer/bytea parameters
- Dominant language
- TypeScript
- Stars
- 655
- Forks
- 97
- PR merge metrics
- No merged PRs in 30d
Description
My app manages deep learning models and it's metadata, the models are sized around 50MB - 100MB.
I am using CockroachDB for storage because it has no hard limit on their `BYTES / BYTEA / BLOB` columns, it is handy when the binary data is infrequently accessed. The binary data is for records and statistical study, they are usually written once and rarely read.
I know it is not the best practice out there, but I don't want to setup object storage just for them.
With the current version, when the buffer goes more than ~30MB V8 will start throwing heap allocation failures, I believe https://github.com/denoland/rusty_v8/pull/427 is not enough to catch up with the sudden memory spike.
Does it make sense if the parameters can be transferred via `ReadableStream` and `WriteableStream`?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.