GoogleCloudPlatform / GoogleCloudPlatform/cloud-spanner-emulator

WHERE IN subqueries appear to cause a CPU spike/are slow

Open
#209 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
334
Forks
77
Avg merge
8m
Merged PRs (30d)
2

Description

I was working on some queries with the local spanner emulator, and noticed that certain queries (namely ones that use something like `WHERE column IN (select id from other_table)`) were returning unexpectedly slowly, as well as causing a CPU spike in my local Spanner Emulator docker container.

My basic reproduction table schema:
```sql
CREATE TABLE IF NOT EXISTS `parent` (
`id` STRING (36) NOT NULL DEFAULT (GENERATE_UUID()),
`name` STRING(MAX) NOT NULL
) PRIMARY KEY (`id`);

CREATE TABLE IF NOT EXISTS `child` (
`id` STRING(36) NOT NULL,
`child_id` STRING(36) NOT NULL DEFAULT(GENERATE_UUID()),
`other_parent_id` STRING(36) NOT NULL,

FOREIGN KEY (`other_parent_id`) REFERENCES `parent` (`id`)
) PRIMARY KEY (`id`, `child_id`),
INTERLEAVE IN PARENT `parent` ON DELETE CASCADE;
```

After inserting ~1000 rows of seed data, running the following query takes about ~3s locally, and <200ms on a Cloud Spanner instance:

```sql
SELECT * FROM `parent` WHERE `id` IN (SELECT `id` FROM `child` WHERE `other_parent_id` = "test-id")
```

In my actual code that I first noticed this issue, I regularly saw similar queries using WHERE IN taking >20s to resolve with the emulator, and less than a second on Cloud Spanner. The local emulator would also spike up to over 100% CPU when running these queries.

Full Test Script (Node.js)

```js
import { Spanner } from "@google-cloud/spanner";
import { randomUUID } from "node:crypto";

const client = new Spanner({
projectId: process.env.GCP_PROJECT_ID || "test-project",
});

const randomId = () => Math.floor(Math.random() * 10000).toString();

const isEmulator = Boolean(process.env.SPANNER_EMULATOR_HOST);
const instanceId = process.env.GCP_INSTANCE_ID || `instance-${randomId()}`;
const databaseId = process.env.GCP_DATABASE_ID || `database-${randomId()}`;

const db = client.instance(instanceId).database(databaseId);

if (isEmulator) {
console.log(`creating test instance ${instanceId}...`);
await client.createInstance(instanceId, {
config: "emulator-config",
description: "test instance",
nodes: 1,
});

console.log(`creating test database ${databaseId}...`);
await client.instance(instanceId).createDatabase(databaseId);
}

const [parentOp] = await db.updateSchema(
`
CREATE TABLE IF NOT EXISTS \`parent\` (
\`id\` STRING (36) NOT NULL DEFAULT (GENERATE_UUID()),
\`name\` STRING(MAX) NOT NULL
) PRIMARY KEY (\`id\`)
`,
);
await parentOp.promise();

const [childOp] = await db.updateSchema(
`
CREATE TABLE IF NOT EXISTS \`child\` (
\`id\` STRING (36) NOT NULL,
\`child_id\` STRING (36) NOT NULL,
\`other_parent_id\` STRING (36) NOT NULL,

FOREIGN KEY (\`other_parent_id\`) REFERENCES \`parent\` (\`id\`)
) PRIMARY KEY (\`id\`, \`child_id\`),
INTERLEAVE IN PARENT \`parent\` ON DELETE CASCADE
`,
);
await childOp.promise();

const parents = Array.from({ length: 1000 }, (_, i) => ({
id: randomUUID(),
name: `parent-${i}`,
}));
const children = Array.from({ length: 1000 }, (_, i) => ({
id: parents[i].id,
child_id: randomUUID(),
other_parent_id: parents[Math.floor(Math.random() * parents.length)].id,
}));

console.time("inserts");
await db.runTransactionAsync(async (tx) => {
tx.insert("parent", parents);
tx.insert("child", children);

await tx.commit();
});
console.timeEnd("inserts");

console.time("query");

const [rows] = await db.run({
sql: `SELECT * FROM \`parent\` WHERE \`id\` IN (SELECT \`id\` FROM \`child\` WHERE \`other_parent_id\` = "${parents[0].id}")`,
json: true,
});
console.timeEnd("query");
console.log(rows);
```

CPU Spike with `docker stats`:

https://github.com/user-attachments/assets/633af134-44c9-4592-9c8c-d9664409a740

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.