gcanti / gcanti/parser-ts

Support for NodeJS Buffer

Open
#55 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
196
Forks
19
PR merge metrics
No merged PRs in 30d

Description

## 🚀 Feature request

Firstly, great work on this lib and `fp-ts` in general. Definitely has changed the way I think about writing software big time. Secondly, why I'm here, hoping you can help me use `parser-ts` more easily/efficiently for binary messages in a Node environment.

Any help or advice greatly appreciated!

### Current Behavior

Currently it's not easy to use a `Stream` with a buffer of the Node `Buffer` type. I'm able to make it work by casting to an `Array` on construction, which works fine since the lib code is only doing lookups, length checks and slices I believe, all of which are supported by `Buffer` and/or its parent interface `Uint8Array`.

### Desired Behavior

It'd be nice to support the `Buffer` type also. It'd also be nice to have `getMany`/ `getManyAndNext` functions for extracting a slice of a buffer. For example in the wire protocol I'm looking at (PostgreSQL) some messages have 32 bit big endian length prefixes which indicate the length of the buffer following that pertain to that message.

### Suggested Solution

I've been able to get some stuff working with a bit of casting here and there. Here's a test program to demonstrate:

```
import { pipe } from 'fp-ts/lib/function';
import * as O from 'fp-ts/lib/Option';
import * as E from 'fp-ts/lib/Either';
import * as P from 'parser-ts/Parser';
import * as PR from 'parser-ts/ParseResult';
import * as S from 'parser-ts/Stream';

type Byte = number;

const stream: (buf: Buffer, cursor?: number) => S.Stream = (
buf,
cursor
) => S.stream(buf as unknown as Array, cursor); // Buffer behaves enough like an Array for use by parser-ts - that is, supports: slice, at/[], length

function buffer(i: Buffer): P.Parser;
function buffer(i: string, enc?: BufferEncoding): P.Parser;
function buffer(
i: string | Buffer,
enc?: BufferEncoding
): P.Parser {
let buf: Buffer;
if (typeof i === 'string') {
buf = Buffer.from(i, enc);
} else {
buf = i;
}
return P.expected(
P.ChainRec.chainRec(buf, (acc) =>
pipe(
O.fromNullable(acc.at(0)),
O.fold(
() => P.of(E.right(buf)),
(c) =>
pipe(
P.sat((b: Byte) => b === c),
P.chain(() => P.of(E.left(acc.slice(1))))
)
)
)
),
JSON.stringify(buf)
);
}

const getManyAndNext: (
i: S.Stream
,
count: number
) => O.Option<{
value: A[];
next: S.Stream
;
}> = (i, count) => {
const endIndex = count + i.cursor;
if (endIndex <= i.buffer.length) {
return O.some({
value: i.buffer.slice(i.cursor, endIndex), // our buffer using Buffer actually outputs a Buffer here, not an Array
next: S.stream(i.buffer, endIndex)
});
}
return O.none;
};

const items: (count: number) => P.Parser> =
(count: number) => (i) =>
pipe(
getManyAndNext(i, count),
O.fold(
() => PR.error(i),
({ value, next }) => PR.success(value, next, i)
)
);

const uint32BE = pipe(
items(4),
P.map((buf) => Buffer.from(buf).readUInt32BE()) // not ideal - creating a buf here is not necessary
// P.map((buf) => (buf as unknown as Buffer).readUInt32BE()) // this is more efficient/ works too but relies on casting
);

const bufParser = pipe(
buffer('x'),
P.chain(() => uint32BE),
P.chain((n) => items(n))
);

const buf = Buffer.allocUnsafe(11);
buf.write('x');
buf.writeUint32BE(5, 1);
buf.writeUint8(1, 5);
buf.writeUint8(2, 6);
buf.writeUint8(3, 7);
buf.writeUint8(4, 8);
buf.writeUint8(5, 9);
buf.writeUint8(6, 10);

console.log(bufParser(stream(buf)));

// Output:
// {
// _tag: 'Right',
// right: {
// value: ,
// next: { buffer: , cursor: 10 },
// start: { buffer: , cursor: 0 }
// }
// }
```

### Who does this impact? Who is this for?

This is for being able to create a parser that deals easily and efficiently with binary content.

| Software | Version(s) |
| ---------- | ---------- |
| fp-ts | 2.11.9 |
| parser-ts | 0.6.16 |
| TypeScript | 4.5.2 |

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.