drizzle-team / drizzle-team/drizzle-orm

[FEATURE]: SQLite: Support `blob` SQL type in relational queries

Open
#2,171 0 comments 2 reactions 0 assignees View on GitHub
enhancement
Dominant language
TypeScript
Stars
35.8k
Forks
1.6k
Avg merge
2d 7h
Merged PRs (30d)
4

Description

### Describe what you want

This issue has been made a few times and the conclusion is store json in text not blob type. However, my use case is storing UUIDs in blob instead of text to cut down on storage. (Maybe there's perf gains for column comparisons too, who knows)

Proposed solution:

If the column is of blob type instead of sending an invalid query, wrap the field in `hex` and return it as a string then convert that back into a `Buffer` in drizzle.

The first part seems easy,
```ts
field.getSQLType() === 'blob' ? sql.raw(`hex("${field.name}")`)
```
If this could be done better, let me know.

Git Diff

```diff
diff --git a/drizzle-orm/src/sqlite-core/dialect.ts b/drizzle-orm/src/sqlite-core/dialect.ts
index aa229d23..ec485a31 100644
--- a/drizzle-orm/src/sqlite-core/dialect.ts
+++ b/drizzle-orm/src/sqlite-core/dialect.ts
@@ -638,7 +638,13 @@ export abstract class SQLiteDialect {
let field = sql`json_array(${
sql.join(
selection.map(({ field }) =>
- is(field, SQLiteColumn) ? sql.identifier(field.name) : is(field, SQL.Aliased) ? field.sql : field
+ is(field, SQLiteColumn)
+ ? field.getSQLType() === 'blob'
+ ? sql.raw(`hex("${field.name}")`)
+ : sql.identifier(field.name)
+ : is(field, SQL.Aliased)
+ ? field.sql
+ : field
),
sql`, `,
)
```

Converting this string back to a buffer seems less easy to me.

Maybe we can add:
```ts
mapFromDriverValue(value: Buffer | string) {
if (typeof value === "string") {
value = hexStringToBuffer(value);
}
return value;
}
```
To `SQLiteBlobBuffer`. And then `value.toString()` -> `typeof value === "string" ? value : value.toString()` in the other two blob classes.

That just leaves custom column types:

```ts
override mapFromDriverValue(value: T['driverParam']): T['data'] {
+ if (typeof value === 'string' && this.getSQLType() === 'blob') {
+ value = hexStringToBuffer(value);
+ }
return typeof this.mapFrom === 'function' ? this.mapFrom(value) : value as T['data'];
}
```

Cons:
Extra overhead of converting hex strings to Buffer. This isn't done for BlobJSON or BlobBigInt but it is always done for custom blob columns and regular BlobBuffer even if the consumer then converts these to strings. So there's some compute wasted there.
Pros:
We can fetch blob type columns in relational queries!

Many thanks for reading :heart:

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.