googleapis / googleapis/google-cloud-ruby

Bigtable: read_rows return extra rows when DeadlineExceed during enumerate

Open
#19,112 0 comments 1 reaction 0 assignees View on GitHub
api: bigtable priority: p3 type: bug
Dominant language
Ruby
Stars
1.4k
Forks
578
Avg merge
1d 11h
Merged PRs (30d)
166

Description

#### Environment details

- OS: debian
- Ruby version: 2.7.6
- Gem name and version: google-cloud-bigtable 2.7.0

#### Steps to reproduce

https://github.com/googleapis/google-cloud-ruby/pull/19111

#### Problem

when pass in `row_keys`, SDK could return extra rows, IF there is error (e.g. DeadlineExceed) occur during processing the last element in enumerator

#### Investigation

I investigate a bit and here is my thoughts

1. when timeout happen during processing the last element, it will retry with new `row_set`

https://github.com/googleapis/google-cloud-ruby/blob/a3e9292e8046ac9f5f7c8e9eca8c576840f1c9ef/google-cloud-bigtable/lib/google/cloud/bigtable/read_operations.rb#L157-L164

2. it smartly prune keys up to `last_key`, so the new `row_set.row_keys` is `[]`

https://github.com/googleapis/google-cloud-ruby/blob/a3e9292e8046ac9f5f7c8e9eca8c576840f1c9ef/google-cloud-bigtable/lib/google/cloud/bigtable/rows_reader.rb#L148-L149

3. Bigtable server does not return empty array when request `row_keys: []`, however, it returns next available rows, thus SDK receive extra rows

#### Impact

I believe this bug also affect other languages. e.g. Python

https://github.com/googleapis/python-bigtable/blob/49b780d1816df9a83ba0748e33edb1f6b6543758/google/cloud/bigtable/row_data.py#L213-L218

#### Temp workaround

table.read_rows(keys: [xxx]).`to_a`.each

#### Notes on CI test

1. samples acceptance test is disabled by default
2. `assert_match` was used and will still be GREEN even if actual output contains extra rows. shall we use `assert_equal` instead?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.