googleapis / googleapis/google-cloud-ruby
Bigtable: read_rows return extra rows when DeadlineExceed during enumerate
- Dominant language
- Ruby
- Stars
- 1.4k
- Forks
- 578
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 166
Description
#### Environment details
- OS: debian
- Ruby version: 2.7.6
- Gem name and version: google-cloud-bigtable 2.7.0
#### Steps to reproduce
https://github.com/googleapis/google-cloud-ruby/pull/19111
#### Problem
when pass in `row_keys`, SDK could return extra rows, IF there is error (e.g. DeadlineExceed) occur during processing the last element in enumerator
#### Investigation
I investigate a bit and here is my thoughts
1. when timeout happen during processing the last element, it will retry with new `row_set`
https://github.com/googleapis/google-cloud-ruby/blob/a3e9292e8046ac9f5f7c8e9eca8c576840f1c9ef/google-cloud-bigtable/lib/google/cloud/bigtable/read_operations.rb#L157-L164
2. it smartly prune keys up to `last_key`, so the new `row_set.row_keys` is `[]`
https://github.com/googleapis/google-cloud-ruby/blob/a3e9292e8046ac9f5f7c8e9eca8c576840f1c9ef/google-cloud-bigtable/lib/google/cloud/bigtable/rows_reader.rb#L148-L149
3. Bigtable server does not return empty array when request `row_keys: []`, however, it returns next available rows, thus SDK receive extra rows
#### Impact
I believe this bug also affect other languages. e.g. Python
https://github.com/googleapis/python-bigtable/blob/49b780d1816df9a83ba0748e33edb1f6b6543758/google/cloud/bigtable/row_data.py#L213-L218
#### Temp workaround
table.read_rows(keys: [xxx]).`to_a`.each
#### Notes on CI test
1. samples acceptance test is disabled by default
2. `assert_match` was used and will still be GREEN even if actual output contains extra rows. shall we use `assert_equal` instead?
Contributor guide
Assessment
This issue has not been assessed yet.