GoogleCloudPlatform / GoogleCloudPlatform/cloud-spanner-emulator
WHERE IN subqueries appear to cause a CPU spike/are slow
- Dominant language
- C++
- Stars
- 334
- Forks
- 77
- Avg merge
- 8m
- Merged PRs (30d)
- 2
Description
I was working on some queries with the local spanner emulator, and noticed that certain queries (namely ones that use something like `WHERE column IN (select id from other_table)`) were returning unexpectedly slowly, as well as causing a CPU spike in my local Spanner Emulator docker container.
My basic reproduction table schema:
```sql
CREATE TABLE IF NOT EXISTS `parent` (
`id` STRING (36) NOT NULL DEFAULT (GENERATE_UUID()),
`name` STRING(MAX) NOT NULL
) PRIMARY KEY (`id`);
CREATE TABLE IF NOT EXISTS `child` (
`id` STRING(36) NOT NULL,
`child_id` STRING(36) NOT NULL DEFAULT(GENERATE_UUID()),
`other_parent_id` STRING(36) NOT NULL,
FOREIGN KEY (`other_parent_id`) REFERENCES `parent` (`id`)
) PRIMARY KEY (`id`, `child_id`),
INTERLEAVE IN PARENT `parent` ON DELETE CASCADE;
```
After inserting ~1000 rows of seed data, running the following query takes about ~3s locally, and <200ms on a Cloud Spanner instance:
```sql
SELECT * FROM `parent` WHERE `id` IN (SELECT `id` FROM `child` WHERE `other_parent_id` = "test-id")
```
In my actual code that I first noticed this issue, I regularly saw similar queries using WHERE IN taking >20s to resolve with the emulator, and less than a second on Cloud Spanner. The local emulator would also spike up to over 100% CPU when running these queries.
Full Test Script (Node.js)
```js
import { Spanner } from "@google-cloud/spanner";
import { randomUUID } from "node:crypto";
const client = new Spanner({
projectId: process.env.GCP_PROJECT_ID || "test-project",
});
const randomId = () => Math.floor(Math.random() * 10000).toString();
const isEmulator = Boolean(process.env.SPANNER_EMULATOR_HOST);
const instanceId = process.env.GCP_INSTANCE_ID || `instance-${randomId()}`;
const databaseId = process.env.GCP_DATABASE_ID || `database-${randomId()}`;
const db = client.instance(instanceId).database(databaseId);
if (isEmulator) {
console.log(`creating test instance ${instanceId}...`);
await client.createInstance(instanceId, {
config: "emulator-config",
description: "test instance",
nodes: 1,
});
console.log(`creating test database ${databaseId}...`);
await client.instance(instanceId).createDatabase(databaseId);
}
const [parentOp] = await db.updateSchema(
`
CREATE TABLE IF NOT EXISTS \`parent\` (
\`id\` STRING (36) NOT NULL DEFAULT (GENERATE_UUID()),
\`name\` STRING(MAX) NOT NULL
) PRIMARY KEY (\`id\`)
`,
);
await parentOp.promise();
const [childOp] = await db.updateSchema(
`
CREATE TABLE IF NOT EXISTS \`child\` (
\`id\` STRING (36) NOT NULL,
\`child_id\` STRING (36) NOT NULL,
\`other_parent_id\` STRING (36) NOT NULL,
FOREIGN KEY (\`other_parent_id\`) REFERENCES \`parent\` (\`id\`)
) PRIMARY KEY (\`id\`, \`child_id\`),
INTERLEAVE IN PARENT \`parent\` ON DELETE CASCADE
`,
);
await childOp.promise();
const parents = Array.from({ length: 1000 }, (_, i) => ({
id: randomUUID(),
name: `parent-${i}`,
}));
const children = Array.from({ length: 1000 }, (_, i) => ({
id: parents[i].id,
child_id: randomUUID(),
other_parent_id: parents[Math.floor(Math.random() * parents.length)].id,
}));
console.time("inserts");
await db.runTransactionAsync(async (tx) => {
tx.insert("parent", parents);
tx.insert("child", children);
await tx.commit();
});
console.timeEnd("inserts");
console.time("query");
const [rows] = await db.run({
sql: `SELECT * FROM \`parent\` WHERE \`id\` IN (SELECT \`id\` FROM \`child\` WHERE \`other_parent_id\` = "${parents[0].id}")`,
json: true,
});
console.timeEnd("query");
console.log(rows);
```
CPU Spike with `docker stats`:
https://github.com/user-attachments/assets/633af134-44c9-4592-9c8c-d9664409a740
Contributor guide
Assessment
This issue has not been assessed yet.