Docker container terminates due to 'Invalid string length' error
- Dominant language
- TypeScript
- Stars
- 2.3k
- Forks
- 393
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 36
Description
Hi, I'm trying to insert around 7 million rows of fake data inside of a table storage resource. The rows consists only of several columns and they don't include any complex structures. I'm inserting the data through transactions, each containing 100 `Add` actions. The data is generated through `Bogus` & I've set sensible values for the `PartitionKey` & `RowKey` columns. `Azurite` is spun up as a `Docker` container w/ persistent storage.
Just below the 1 million row count the container terminates due to a runtime error from `/opt/azurite/node_modules/lokijs/src/lokijs.js:1331`:
```powershell
/opt/azurite/node_modules/lokijs/src/lokijs.js:1331
case "normal": return JSON.stringify(this, this.serializeReplacer);
^
RangeError: Invalid string length
at JSON.stringify ()
at Loki.serialize (/opt/azurite/node_modules/lokijs/src/lokijs.js:1331:36)
at Loki.saveDatabaseInternal (/opt/azurite/node_modules/lokijs/src/lokijs.js:2780:66)
at Loki.saveDatabase (/opt/azurite/node_modules/lokijs/src/lokijs.js:2822:12)
at Timeout.autosaveHandleInterval [as _onTimeout] (/opt/azurite/node_modules/lokijs/src/lokijs.js:2923:16)
at listOnTimeout (node:internal/timers:594:17)
at process.processTimers (node:internal/timers:529:7)
```
Going through the container logs I'm not seeing any other errors or warnings, all previous batches resolve with `HTTP 202` status codes.
It doesn't seem to happen at random, it's always around the 930k record mark, up until that it's reasonably fast. I've also looked at the recent issues in https://github.com/techfort/LokiJS, but I've yet to find any similar experiences.
I've also verified that I'm not using any in-memory storage, so I'm not sure what causes this dependency to error out. Also, I've reviewed the data in the transaction batch which causes this, there's nothing out of the ordinary. Artificial delays also don't resolve this issue.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.