denodrivers / denodrivers/postgres

[feature] Support streaming of buffer/bytea parameters

Open
#408 0 comments 1 reaction 0 assignees View on GitHub
help wanted
Dominant language
TypeScript
Stars
655
Forks
97
PR merge metrics
No merged PRs in 30d

Description

My app manages deep learning models and it's metadata, the models are sized around 50MB - 100MB.

I am using CockroachDB for storage because it has no hard limit on their `BYTES / BYTEA / BLOB` columns, it is handy when the binary data is infrequently accessed. The binary data is for records and statistical study, they are usually written once and rarely read.

I know it is not the best practice out there, but I don't want to setup object storage just for them.

With the current version, when the buffer goes more than ~30MB V8 will start throwing heap allocation failures, I believe https://github.com/denoland/rusty_v8/pull/427 is not enough to catch up with the sudden memory spike.

Does it make sense if the parameters can be transferred via `ReadableStream` and `WriteableStream`?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.