isomorphic-git / isomorphic-git/lightning-fs

How to speedup writeFile?

Open
#115 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
610
Forks
62
Avg merge
14h 47m
Merged PRs (30d)
1

Description

I have a code to read a real folder from File Access API into lightningFS.

The repo is about 800kb, has 300 files.

The code works very slowly, because of writeFile (~1.3 sec).

I've read that indexedDB throttles `writeFile`, is that so?
How to speedup?

Here's the code, it recursively reads all dirs/files and uses `fs.promises.writeFile` to write them to lightningFS.

This `writeFile` call is the main reason for the delay, even though the data is very small.

```js run
let fs = new LightningFS('fs', {wipe: true});

let relPath = [''];

async function handle(dirHandle) {
console.time(dirHandle.name);

for await (const entry of dirHandle.values()) {
if (entry.kind === "file") {
const file = await entry.getFile();
let data = new Uint8Array(await file.arrayBuffer());
let filePath = [...relPath, file.name].join('/');
await fs.promises.writeFile(filePath, data);
}

if (entry.kind === "directory") {
const newHandle = await dirHandle.getDirectoryHandle( entry.name, { create: false } );
relPath.push(entry.name);
let dirPath = relPath.join('/');
await fs.promises.mkdir(dirPath);
await handle(newHandle);
relPath.pop();
}
}
}
```

P.S. Is there any other backend for lightningFS? I need a simple in-memory strorage.
It's quite ironic that lightningFS is so sluggish for a tiny test repo.
Maybe I'm doing something wrong?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.