Shopify / Shopify/identity_cache

Thread safety when calling fetch_relation

Open
#568 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Ruby
Stars
2k
Forks
174
Avg merge
13m
Merged PRs (30d)
3

Description

Hi!

We utilize a global, in-memory cache with TTL for one of our models per process. We also wanted to use identity_cache so that the whole object tree is fetched with one network call.

However, we receive rare errors when we call fetch_association with the following stack trace:

NameError instance variable @dehydrated_relation not defined

/usr/local/bundle/ruby/3.3.0/gems/identity_cache-1.5.6/lib/identity_cache/cached/recursive/association.rb:34
/usr/local/bundle/ruby/3.3.0/gems/identity_cache-1.5.6/lib/identity_cache/cached/recursive/association.rb:19

which points to this piece of code:

https://github.com/Shopify/identity_cache/blob/313dfbcca210fdadd739268cfdcce01ff854f0b4/lib/identity_cache/cached/recursive/association.rb#L25-L42

Since the code has a guard clause: record.instance_variable_defined?(dehydrated_variable_name), the only way this can happen is that another thread is concurrently executing the same code and has already removed this instance variable.

Are you open to contributions to fix concurrency issues? So far we've only seen this error popping up dozens of times on ~50M requests, however, there might be more that we haven't seen yet or are failing silently 👀

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read lib/identity_cache/cached/recursive/association.rb around lines 25-42, starting with the guard and removal of @dehydrated_relation. Investigate concurrent fetch_association calls and verify that the reported NameError no longer occurs under concurrent access.

Written by the indexing model from the issue text.

Assessment

Tech stack
ruby
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.