Support for NodeJS Buffer
- Dominant language
- TypeScript
- Stars
- 196
- Forks
- 19
- PR merge metrics
- No merged PRs in 30d
Description
## 🚀 Feature request
Firstly, great work on this lib and `fp-ts` in general. Definitely has changed the way I think about writing software big time. Secondly, why I'm here, hoping you can help me use `parser-ts` more easily/efficiently for binary messages in a Node environment.
Any help or advice greatly appreciated!
### Current Behavior
Currently it's not easy to use a `Stream` with a buffer of the Node `Buffer` type. I'm able to make it work by casting to an `Array` on construction, which works fine since the lib code is only doing lookups, length checks and slices I believe, all of which are supported by `Buffer` and/or its parent interface `Uint8Array`.
### Desired Behavior
It'd be nice to support the `Buffer` type also. It'd also be nice to have `getMany`/ `getManyAndNext` functions for extracting a slice of a buffer. For example in the wire protocol I'm looking at (PostgreSQL) some messages have 32 bit big endian length prefixes which indicate the length of the buffer following that pertain to that message.
### Suggested Solution
I've been able to get some stuff working with a bit of casting here and there. Here's a test program to demonstrate:
```
import { pipe } from 'fp-ts/lib/function';
import * as O from 'fp-ts/lib/Option';
import * as E from 'fp-ts/lib/Either';
import * as P from 'parser-ts/Parser';
import * as PR from 'parser-ts/ParseResult';
import * as S from 'parser-ts/Stream';
type Byte = number;
const stream: (buf: Buffer, cursor?: number) => S.Stream = (
buf,
cursor
) => S.stream(buf as unknown as Array, cursor); // Buffer behaves enough like an Array for use by parser-ts - that is, supports: slice, at/[], length
function buffer(i: Buffer): P.Parser;
function buffer(i: string, enc?: BufferEncoding): P.Parser;
function buffer(
i: string | Buffer,
enc?: BufferEncoding
): P.Parser {
let buf: Buffer;
if (typeof i === 'string') {
buf = Buffer.from(i, enc);
} else {
buf = i;
}
return P.expected(
P.ChainRec.chainRec(buf, (acc) =>
pipe(
O.fromNullable(acc.at(0)),
O.fold(
() => P.of(E.right(buf)),
(c) =>
pipe(
P.sat((b: Byte) => b === c),
P.chain(() => P.of(E.left(acc.slice(1))))
)
)
)
),
JSON.stringify(buf)
);
}
const getManyAndNext: (
i: S.Stream,
count: number
) => O.Option<{
value: A[];
next: S.Stream;
}> = (i, count) => {
const endIndex = count + i.cursor;
if (endIndex <= i.buffer.length) {
return O.some({
value: i.buffer.slice(i.cursor, endIndex), // our buffer using Buffer actually outputs a Buffer here, not an Array
next: S.stream(i.buffer, endIndex)
});
}
return O.none;
};
const uint32BE = pipe(
items(4),
P.map((buf) => Buffer.from(buf).readUInt32BE()) // not ideal - creating a buf here is not necessary
// P.map((buf) => (buf as unknown as Buffer).readUInt32BE()) // this is more efficient/ works too but relies on casting
);
const bufParser = pipe(
buffer('x'),
P.chain(() => uint32BE),
P.chain((n) => items(n))
);
const buf = Buffer.allocUnsafe(11);
buf.write('x');
buf.writeUint32BE(5, 1);
buf.writeUint8(1, 5);
buf.writeUint8(2, 6);
buf.writeUint8(3, 7);
buf.writeUint8(4, 8);
buf.writeUint8(5, 9);
buf.writeUint8(6, 10);
console.log(bufParser(stream(buf)));
// Output:
// {
// _tag: 'Right',
// right: {
// value: ,
// next: { buffer: , cursor: 10 },
// start: { buffer: , cursor: 0 }
// }
// }
```
### Who does this impact? Who is this for?
This is for being able to create a parser that deals easily and efficiently with binary content.
| Software | Version(s) |
| ---------- | ---------- |
| fp-ts | 2.11.9 |
| parser-ts | 0.6.16 |
| TypeScript | 4.5.2 |
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.