Cannot parse from a stream in the browser
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 4.3k
- Forks
- 299
- Avg merge
- 16h 19m
- Merged PRs (30d)
- 1
Description
Describe the bug
I want to allow a user to upload a gzipped csv file and parse it in a stream (using pipeThrough / pipeTo). This works in Node, but not in the browser, where I get:
file-upload.html:52 Uncaught (in promise) TypeError: Failed to execute 'pipeTo' on 'ReadableStream': parameter 1 is not of type 'WritableStream'.
at HTMLInputElement.<anonymous> (file-upload.html:52:18)
Line 52 is the pipeTo line from the example below, so it seems to think that the parse object is not a WritableStream.
To Reproduce
Dump the code below into a file, open it in a browser and then try uploading a .tsv.gz file. I tested in Chrome and Firefox. The same code works in Node, although you have to swap pipeThrough and pipeTo for pipe. I'll admit that I find the differences between Node and the browser for streaming a bit confusing so I may have made some obvious error here.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>File Upload</title>
</head>
<body>
<div class="upload-container">
<form class="upload-form">
<input type="file" id="fileInput" name="file">
</form>
</div>
<script type="module">
import { parse } from 'https://cdn.jsdelivr.net/npm/csv-parse@5.5.6/+esm'
document.querySelector('#fileInput').addEventListener('change', (e) => {
e.preventDefault();
const file = document.getElementById('fileInput').files[0];
// turn the file into a stream
const fileStream = file.stream()
// create a gunzip transform stream
const unzipper = new DecompressionStream('gzip')
const records = [];
// create a csv parser writeable stream
const parser = parse({
delimiter: '\t'
});
parser.on('readable', function(){
let record;
while ((record = parser.read()) !== null) {
records.push(record);
}
});
parser.on('error', function(err){
console.error(err.message);
});
parser.on('end', function(){
console.log(records)
})
console.log("piping!")
// pipe the file stream through the gunzip stream and into the csv parse stream
fileStream
.pipeThrough(unzipper)
.pipeTo(parser)
});
</script>
</body>
</html>
** Additional Context **
I wonder if this issue is related, although I'm not trying to parse the complex CSVs listed in that issue.
I also have a similar issue raised for Papaparse.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the browser example in file-upload.html, especially line 52 where the file stream is piped into the parser, and compare the parser export with the browser WritableStream contract. Reproduce the upload in Chrome or Firefox and verify that a gzipped TSV reaches the parser and emits the expected records without the pipeTo type error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js
- Domain
- stream-processing, web-dev
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100