GoogleCloudPlatform / GoogleCloudPlatform/cloud-spanner-emulator

Per-table limit on total size of non-key columns (2000 MiB) is not enforced

Open
#361 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
334
Forks
77
Avg merge
8m
Merged PRs (30d)
2

Description

The emulator does not enforce Cloud Spanner's per-table limit on the total size of non-key columns (2,097,152,000 bytes = 2000 MiB). DDL that exceeds the limit succeeds on the emulator but fails on Cloud Spanner, in all three paths: `CREATE TABLE`, `ALTER TABLE ... ADD COLUMN`, and `ALTER TABLE ... ALTER COLUMN`.

Per the [quotas documentation](https://docs.cloud.google.com/spanner/quotas#tables):
> Total size of non-key columns | 2,000 MiB

### Reproduction

Image: `gcr.io/cloud-spanner-emulator/emulator:latest` (digest `sha256:ad54472fe7b161b9214f7f816f304b649a4779e348229c375ac067f5ed5a6422`, corresponds to `v1.5.55`).

Setup — two tables whose non-key columns are at or under the limit. Both the emulator and Cloud Spanner accept these (200 × `STRING(MAX)` is exactly 2,097,152,000 bytes, so the boundary itself is fine):

```sql
-- Exactly at the limit: 200 STRING(MAX) columns
CREATE TABLE T200 (
Id STRING(36) NOT NULL,
Col001 STRING(MAX),
-- ... 200 STRING(MAX) columns in total ...
Col200 STRING(MAX),
) PRIMARY KEY (Id);

-- Under the limit: 199 STRING(MAX) columns + 2 STRING(100) columns
CREATE TABLE TAlter (
Id STRING(36) NOT NULL,
Col001 STRING(MAX),
-- ... 199 STRING(MAX) columns in total ...
Col199 STRING(MAX),
Extra1 STRING(100),
Extra2 STRING(100),
) PRIMARY KEY (Id);
```

Statements that exceed the limit:

```sql
-- (A) CREATE TABLE with 201 STRING(MAX) columns
CREATE TABLE T201 (
Id STRING(36) NOT NULL,
Col001 STRING(MAX),
-- ... 201 STRING(MAX) columns in total ...
Col201 STRING(MAX),
) PRIMARY KEY (Id);

-- (B) Add a column to the at-limit table
ALTER TABLE T200 ADD COLUMN Col201 STRING(MAX);

-- (C) Widen an existing column
ALTER TABLE TAlter ALTER COLUMN Extra1 STRING(MAX);
```

### Expected (Cloud Spanner behavior)

All three statements fail:

```
(A) ERROR: Total size of non-key columns in Table T201 2107637760 exceeds the limit of 2097152000.
(B) ERROR: Cannot add column Col201. Total size of non-key columns in table T200 2107637760 exceeds the limit of 2097152000.
(C) ERROR: Cannot increase length of column Extra1. Total size of non-key columns in table TAlter is 2097152400 which exceeds the limit of 2097152000.
```

### Actual (Emulator behavior)

All three statements succeed, and the resulting schema contains the over-limit tables (`T201`, `T200` with `Col201`, and `TAlter` with `Extra1 STRING(MAX)`).

### Impact

CI that runs DDL against the emulator passes, but the same DDL fails when applied to Cloud Spanner — typically discovered only at migration time, when a wide table (e.g. one with many `STRING(MAX)` columns) crosses the 2000 MiB budget.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the emulator's DDL validation for CREATE TABLE, ALTER TABLE ... ADD COLUMN, and ALTER TABLE ... ALTER COLUMN. Reproduce the three over-limit statements and compare them with the accepted at-limit cases, then add coverage showing that each path rejects totals above 2,097,152,000 bytes while accepting the boundary.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
database
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.