tursodatabase / tursodatabase/libsql
Insert queries take an enormous amount of time with replicas
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 17.2k
- Forks
- 531
- Avg merge
- 1h 12m
- Merged PRs (30d)
- 1
Description
tursoClient.js
import { createClient } from "@libsql/client"
const tursoClient = createClient({
url: process.env.TURSO_DATABASE_URL,
authToken: process.env.TURSO_AUTH_TOKEN,
})
export default tursoClient
app.js
import tursoClient from './lib/tursoClient.js'
async function testDB() {
const iterations = 1000 // Number of iterations for stress testing
// Variables to store performance metrics
let totalQueryTime = 0
let totalWriteTime = 0
// Read operation
for (let i = 0; i < iterations; i++) {
const startQueryTime = Date.now()
const employees = await tursoClient.execute("SELECT * FROM Employees")
const endQueryTime = Date.now()
totalQueryTime += endQueryTime - startQueryTime
}
// Write operation (inserting a new employee)
for (let i = 0; i < iterations; i++) {
const startWriteTime = Date.now()
// Generate random data for the new employee
const newEmployeeFirstName = await getRandomString()
const newEmployeeLastName = await getRandomString()
const newEmployeeDepartmentID = 1
// Execute the INSERT query
await tursoClient.execute({
sql: "INSERT INTO Employees (FirstName, LastName, DepartmentID) VALUES (?, ?, ?)",
args: [newEmployeeFirstName, newEmployeeLastName, newEmployeeDepartmentID]
})
const endWriteTime = Date.now()
totalWriteTime += endWriteTime - startWriteTime
}
// Print performance reports
const averageQueryTime = totalQueryTime / iterations
console.log(`Average query time for SELECT operation: ${averageQueryTime} ms`)
const averageWriteTime = totalWriteTime / iterations
console.log(`Average time taken for INSERT operation: ${averageWriteTime} ms`)
}
async function getRandomString() {
const characters = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz'
let randomString = ''
for (let i = 0; i < 5; i++) {
randomString += characters.charAt(Math.floor(Math.random() * characters.length))
}
return randomString
}
await testDB()
Results:
Primary only:
root@ubuntu-s-4vcpu-16gb-amd-ams3-01:~/pastebin# node --env-file=.env app.js
Average query time for SELECT operation: 20.641 ms
Average time taken for INSERT operation: 19.172 ms
Primary + 2 replicas:
root@ubuntu-s-4vcpu-16gb-amd-ams3-01:~/pastebin# node --env-file=.env app.js
Average query time for SELECT operation: 13.753 ms
Average time taken for INSERT operation: 132.646 ms
Is this acceptable? Is there a way to disable replication acknowledgment for insert operations, similar to MongoDB?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the reproduction in tursoclient.js and app.js, then run node --env-file=.env app.js against the primary-only and primary-plus-replicas configurations described in the report. Trace the insert path and replication acknowledgment behavior to determine whether the latency is expected and whether an acknowledgment option is feasible; document the conclusion and any required design changes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js, sqlite
- Domain
- databases, distributed-systems, performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100