googleapis / googleapis/ruby-spanner-activerecord

Floating point serialization issues in JSON fields

Đang mở
#376 0 bình luận 1 reaction 1 người được giao Được @olavloite nhận Xem trên GitHub
api: spanner priority: p2 type: bug
Ngôn ngữ chính
Ruby
Star
105
Fork
37
Merge trung bình
10 giờ 2 phút
Pull request đã merge (30 ngày)
5

Mô tả

We've recently run into some issues serializing specific floating point numbers. We have hit these before, and our understanding is that they stem from Ruby using a different floating point serialization algorithm from Spanner (which spanner runs some API-layer checks to validate).

Prior to this, we were able to work around them by monkeypatching `Float#to_s` to use a matching grisu2 implementation from [2].

Unfortunately after `v2.11.0` of `ruby/json`[1], this workaround no longer works. `ruby/json` no longer delegates the serialization back to `Float#to_s` and instead uses it's own C implementation of `grisu2`. It's worth noting this isn't a new issue, but has become much more challenging to monkeypatch away now that it doesn't call back to the Ruby layer.

Our understanding is Spanner also uses grisu2 internally, however there seems to be some inconsistencies with implementations in [2] and [1]. In an ideal scenario, the SDK with should maintain a consistent serialization interface with the validations performed by the API layer.

#### Steps to reproduce

1. Add the following to `acceptance/cases/type/json_test.rb`:
```
def test_float_serialization
record = TestTypeModel.create! details: { test: 41.021725 }
record.reload

assert_equal 41.021725, record.details[:test]
end
```
2. Run the test against a remote spanner instance (it passes on the emulator).
3. The test should `error` with with:
```
1) Error:
ActiveRecord::Type::DateTest#test_float_serialization:
ActiveRecord::StatementInvalid: Google::Cloud::FailedPreconditionError: 9:Invalid value for column details in table test_types: Expected JSON.. debug_error_string:{UNKNOWN:Error received from peer ipv6:%5B2607:f8b0:4005:80e::200a%5D:443 {grpc_message:"Invalid value for column details in table test_types: Expected JSON.", grpc_status:9}}
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/google-cloud-spanner-v1-1.11.0/lib/google/cloud/spanner/v1/spanner/client.rb:1813:in `rescue in commit'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/google-cloud-spanner-v1-1.11.0/lib/google/cloud/spanner/v1/spanner/client.rb:1775:in `commit'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/google-cloud-spanner-2.28.0/lib/google/cloud/spanner/service.rb:557:in `commit'
lib/spanner_client_ext.rb:29:in `commit_transaction'
lib/activerecord_spanner_adapter/transaction.rb:105:in `commit'
lib/activerecord_spanner_adapter/connection.rb:286:in `commit_transaction'
lib/active_record/connection_adapters/spanner/database_statements.rb:305:in `block in commit_db_transaction'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activesupport-7.1.5.2/lib/active_support/notifications/instrumenter.rb:58:in `instrument'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.5.2/lib/active_record/connection_adapters/abstract_adapter.rb:1142:in `log'
lib/active_record/connection_adapters/spanner_adapter.rb:327:in `log'
lib/active_record/connection_adapters/spanner/database_statements.rb:304:in `commit_db_transaction'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.5.2/lib/active_record/connection_adapters/abstract/transaction.rb:400:in `commit'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.5.2/lib/active_record/connection_adapters/abstract/transaction.rb:514:in `block in commit_transaction'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activesupport-7.1.5.2/lib/active_support/concurrency/null_lock.rb:9:in `synchronize'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.5.2/lib/active_record/connection_adapters/abstract/transaction.rb:503:in `commit_transaction'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.5.2/lib/active_record/connection_adapters/abstract/transaction.rb:565:in `ensure in block in within_new_transaction'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.5.2/lib/active_record/connection_adapters/abstract/transaction.rb:564:in `block in within_new_transaction'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activesupport-7.1.5.2/lib/active_support/concurrency/null_lock.rb:9:in `synchronize'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.5.2/lib/active_record/connection_adapters/abstract/transaction.rb:532:in `within_new_transaction'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.5.2/lib/active_record/connection_adapters/abstract/database_statements.rb:344:in `transaction'
lib/active_record/connection_adapters/spanner/database_statements.rb:239:in `transaction'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.5.2/lib/active_record/transactions.rb:212:in `transaction'
lib/activerecord_spanner_adapter/base.rb:28:in `create!'
acceptance/cases/type/json_test.rb:42:in `test_float_serialization'
```

The problematic value here is `41.021725` -- passing that in as a root node to the JSON (which is valid JSON) also fails.

[1] https://github.com/ruby/json/commit/7d77415108c1fd0f55543cf0705abcb7c679d17f
[2] https://github.com/nlohmann/json

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.