exceljs / exceljs/exceljs

ExcelJS CSV Write Stream Not Writing with Large Dataset

Open
#2,282 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
15.5k
Forks
2k
PR merge metrics
No merged PRs in 30d

Description

## 🐛 Bug Report

I am using the ExcelJS library in my Node.js project to write a large dataset to a CSV file using the write stream feature. However, I am experiencing an issue where the write stream seems to fail when dealing with large datasets.

This works with small set of data and was able to upload to blob container.

Additional notes: I was using const csvBuffer = await workbook.csv.writeBuffer(); but it consumes a lot of memory while writing the file so i switched to streaming.

Any help would be appreciated.

Lib version: Latest Version

## Steps To Reproduce

```
const ExcelJS = require('exceljs');
const { BlobServiceClient } = require('@azure/storage-blob');
const stream = require('stream');

async function createAndUploadCSV() {
// Create a new workbook and worksheet
const workbook = new ExcelJS.Workbook();
const worksheet = workbook.addWorksheet('Sheet 1');

// Add some sample data to the worksheet
worksheet.columns = [
{ header: 'Name', key: 'name', width: 20 },
{ header: 'Age', key: 'age', width: 10 },
{ header: 'Email', key: 'email', width: 30 }
];

for (let i = 1; i <= 100000; i++) {
worksheet.addRow({ name: `User ${i}`, age: i, email: `user${i}@example.com` });
}

// Create a writable stream
const writableStream = new stream.PassThrough();

// Write workbook to the stream as CSV
await workbook.csv.write(writableStream, { stream: true });

// Create a BlobServiceClient to connect to Azure Blob Storage
const connectionString = '';
const containerName = 'app-reporting';
const blobServiceClient = BlobServiceClient.fromConnectionString(connectionString);
const containerClient = blobServiceClient.getContainerClient(containerName);

// Create a unique name for the blob using a timestamp
const blobName = `data-${Date.now()}.csv`;

// Upload the CSV stream to Azure Blob Storage
const blockBlobClient = containerClient.getBlockBlobClient(blobName);
const uploadResponse = await blockBlobClient.uploadStream(writableStream);

console.log('File uploaded:', uploadResponse.requestId);
}

createAndUploadCSV().catch(console.error);
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.