sql-js / sql-js/sql.js

out of memory

Open
#568 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
13.7k
Forks
1.1k
PR merge metrics
No merged PRs in 30d

Description

First off, my JavaScript environment is very custom and different than the web or Node.js. We are running in a wasm32-wasi environment with QuickJS, using wasmedge-quickjs, and Wasmtime on the Internet Computer.

That being said, the strange thing is that I had this working about a week ago, but now it doesn't work. We've been doing a lot of low-level changes to our JS environment (we're working at the engine/platform level), but I just don't understand what could have caused this.

The simple problem is that we get an error just trying to initialize the database: out of memory

Here's our code:

import { Server } from 'azle';
import express from 'express';
import initSqlJs from 'sql.js/dist/sql-asm.js';

export default Server(async () => {
    const SQL = await initSqlJs({
        locateFile: (filename) => `/dist/${filename}`
    });

    const db = new SQL.Database();

    const app = express();

    app.get('/', (req, res) => {
        res.send('Hello world');
    });

    return app.listen();
});

We have to use asm as our environment does not have a Wasm interpreter. I've tried sql-asm-memory-growth.js as well and I get the same error. I've even gone back multiple versions to maybe 1.7.0 and the error is always there.

Any ideas on what could cause this? I've also tried adding:

            TOTAL_STACK: 4_000_000_000,
            TOTAL_MEMORY: 4_000_000_000,
            FAST_MEMORY: 4_000_000_000

to the initSqlJs options.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing initialization in the described wasm32-wasi environment, using sql-asm.js and then sql-asm-memory-growth.js. Trace initSqlJs through new SQL.Database() and compare the behavior across the mentioned versions and memory options. Done means identifying a reproducible cause or documenting the environment limitation and a confirmed workaround.

Written by the indexing model from the issue text.

Assessment

Tech stack
express, javascript, sqlite, typescript, wasm
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.