LinusU / LinusU/node-jsonlines

Adding fileStream information to documentation

Open
#6 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement help wanted
Dominant language
JavaScript
Stars
25
Forks
2
PR merge metrics
No merged PRs in 30d

Description

It might be helpful to include some documentation for people who are unfamiliar with transfer streams (like I was until I asked chatGPT). The docs mention that the stringifier writes to a transfer stream, but this is a subtle, and many people are likely to miss that detail or read through it because they don't know what it means.

Here is the code I wrote that can handle file streams:

import jsonlines from "jsonlines"
import fs from "fs"
import path from 'path'

export default function example () {

  // I created a folder in my repo caled "files", and the stringifier will write to it (only locally, don't try this on something like Heroku)
  const outputFilePath = path.join('files', 'jsonl-testing.jsonl')
  const fileStream = fs.createWriteStream(outputFilePath)

  const stringifier = jsonlines.stringify()
  stringifier.pipe(fileStream)
  stringifier.write({
    "prompt": "Question: What is the meaning of life?\n\n###\n\n",
    "completion": "Answer: 42.\n\nEND"
  })
  stringifier.write({
    "prompt": "Question: What is not the meaning of life?\n\n###\n\n",
    "completion": "Answer: not 42.\n\nEND"
  })
  stringifier.end()
  fileStream.end() // do we need to do this? Docs do not specifiy
}

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Locate the documentation describing the stringifier and transfer streams, then compare it with the file-stream example in the issue. Update the documentation so readers understand piping to fs.createWriteStream and whether fileStream.end() is needed; done when the file-stream workflow and stream-closing behavior are clear.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.