googleapis / googleapis/google-cloud-go

spanner: support change streams in spannertest

Open
#14,382 4 comments 0 reactions 1 assignee Claimed by @rahul2393 View on GitHub
api: spanner triage me
Dominant language
Go
Stars
4.5k
Forks
1.6k
Avg merge
1d 13h
Merged PRs (30d)
109

Description

## Is your feature request related to a problem?

We'd like to be able to use spannertest to test change stream implementations without requiring a real Spanner instance or the Cloud Spanner Emulator. Currently, spannertest supports DDL (CREATE CHANGE STREAM) and mutations, but the READ_ TVF used to consume change stream data is not implemented, so any code that queries a change stream cannot be tested with the in-memory fake.

## Describe the solution you'd like

Implement the READ_ TVF in spannertest so that it returns results in the same wire format as real Cloud Spanner:

```
ARRAY>,
heartbeat_record ARRAY>,
child_partitions_record ARRAY>
>>
```

The TVF should support the standard parameters (start_timestamp, end_timestamp, heartbeat_milliseconds) and the partition-based API (partition_token):

- Calling with partition_token => NULL returns a child_partitions_record bootstrapping the single fake partition.
- Calling with the returned token returns data change records grouped by (table, mod_type) per transaction, matching real Spanner's batching behaviour.

Export ready-to-use Go decoder types from the spannertest package so consumers don't need to redeclare them:

```go
var records []*spannertest.ChangeRecord
if err := row.Column(0, &records); err != nil { ... }
for _, cr := range records {
for _, dcr := range cr.DataChangeRecords {
fmt.Println(dcr.TableName, dcr.ModType)
}
}
```

Exported types: ChangeRecord, DataChangeRecord, Mod, ColumnType, HeartbeatRecord, ChildPartition, ChildPartitionsRecord.

## Describe alternatives you've considered

- Cloud Spanner Emulator: The emulator supports change streams fully, but requires Docker and is significantly heavier than spannertest for unit tests. spannertest is preferred for fast, hermetic, dependency-free tests.
- Mocking the Spanner client: Mocking at the client level requires reproducing the full change stream decoding logic and is fragile against client library changes. Testing against a fake server is more realistic.

## Additional context

The implementation is intentionally simplified relative to real Spanner:

- A single fake partition ("spannertest-0") covers all data — multi-partition streaming is not modelled.
- value_capture_type is fixed to NEW_VALUES; OLD_VALUES and NEW_ROW capture modes are not supported.
- DML-sourced mutations (via ExecuteSql) are tracked alongside Apply-sourced mutations.
- Change streams created with column-specific or table-specific watch lists are respected; FOR ALL streams capture all watched tables.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.