googleapis / googleapis/ruby-spanner-activerecord

Floating point serialization issues in JSON fields

オープン
#376 コメント 0 件 リアクション 1 件 担当者 1 名 @olavloite が担当を希望しています GitHub で見る
api: spanner priority: p2 type: bug
主要言語
Ruby
スター
105
フォーク
37
平均マージ
10時間 2分
マージ済み PR(30日)
5

説明

We've recently run into some issues serializing specific floating point numbers. We have hit these before, and our understanding is that they stem from Ruby using a different floating point serialization algorithm from Spanner (which spanner runs some API-layer checks to validate).

Prior to this, we were able to work around them by monkeypatching `Float#to_s` to use a matching grisu2 implementation from [2].

Unfortunately after `v2.11.0` of `ruby/json`[1], this workaround no longer works. `ruby/json` no longer delegates the serialization back to `Float#to_s` and instead uses it's own C implementation of `grisu2`. It's worth noting this isn't a new issue, but has become much more challenging to monkeypatch away now that it doesn't call back to the Ruby layer.

Our understanding is Spanner also uses grisu2 internally, however there seems to be some inconsistencies with implementations in [2] and [1]. In an ideal scenario, the SDK with should maintain a consistent serialization interface with the validations performed by the API layer.

#### Steps to reproduce

1. Add the following to `acceptance/cases/type/json_test.rb`:
```
def test_float_serialization
record = TestTypeModel.create! details: { test: 41.021725 }
record.reload

assert_equal 41.021725, record.details[:test]
end
```
2. Run the test against a remote spanner instance (it passes on the emulator).
3. The test should `error` with with:
```
1) Error:
ActiveRecord::Type::DateTest#test_float_serialization:
ActiveRecord::StatementInvalid: Google::Cloud::FailedPreconditionError: 9:Invalid value for column details in table test_types: Expected JSON.. debug_error_string:{UNKNOWN:Error received from peer ipv6:%5B2607:f8b0:4005:80e::200a%5D:443 {grpc_message:"Invalid value for column details in table test_types: Expected JSON.", grpc_status:9}}
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/google-cloud-spanner-v1-1.11.0/lib/google/cloud/spanner/v1/spanner/client.rb:1813:in `rescue in commit'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/google-cloud-spanner-v1-1.11.0/lib/google/cloud/spanner/v1/spanner/client.rb:1775:in `commit'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/google-cloud-spanner-2.28.0/lib/google/cloud/spanner/service.rb:557:in `commit'
lib/spanner_client_ext.rb:29:in `commit_transaction'
lib/activerecord_spanner_adapter/transaction.rb:105:in `commit'
lib/activerecord_spanner_adapter/connection.rb:286:in `commit_transaction'
lib/active_record/connection_adapters/spanner/database_statements.rb:305:in `block in commit_db_transaction'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activesupport-7.1.5.2/lib/active_support/notifications/instrumenter.rb:58:in `instrument'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.5.2/lib/active_record/connection_adapters/abstract_adapter.rb:1142:in `log'
lib/active_record/connection_adapters/spanner_adapter.rb:327:in `log'
lib/active_record/connection_adapters/spanner/database_statements.rb:304:in `commit_db_transaction'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.5.2/lib/active_record/connection_adapters/abstract/transaction.rb:400:in `commit'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.5.2/lib/active_record/connection_adapters/abstract/transaction.rb:514:in `block in commit_transaction'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activesupport-7.1.5.2/lib/active_support/concurrency/null_lock.rb:9:in `synchronize'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.5.2/lib/active_record/connection_adapters/abstract/transaction.rb:503:in `commit_transaction'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.5.2/lib/active_record/connection_adapters/abstract/transaction.rb:565:in `ensure in block in within_new_transaction'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.5.2/lib/active_record/connection_adapters/abstract/transaction.rb:564:in `block in within_new_transaction'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activesupport-7.1.5.2/lib/active_support/concurrency/null_lock.rb:9:in `synchronize'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.5.2/lib/active_record/connection_adapters/abstract/transaction.rb:532:in `within_new_transaction'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.5.2/lib/active_record/connection_adapters/abstract/database_statements.rb:344:in `transaction'
lib/active_record/connection_adapters/spanner/database_statements.rb:239:in `transaction'
/Users/jarredhawkins/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.5.2/lib/active_record/transactions.rb:212:in `transaction'
lib/activerecord_spanner_adapter/base.rb:28:in `create!'
acceptance/cases/type/json_test.rb:42:in `test_float_serialization'
```

The problematic value here is `41.021725` -- passing that in as a root node to the JSON (which is valid JSON) also fails.

[1] https://github.com/ruby/json/commit/7d77415108c1fd0f55543cf0705abcb7c679d17f
[2] https://github.com/nlohmann/json

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。