Support temporarily disabling persisting nils for a model

Background

We have an odd scenario where we have a model with a default scope.

class Model < ApplicationRecord
  enum create_status: [:in_progress, :success, :failed]
  default_scope { where(create_status: :success) }
end

class ModelBuilder
  def build
    model = Model.new(create_status: :in_progress)
    model.save

    # Do a bunch of work, taking a few seconds either inline or perhaps in a background job, then:
    model.update(create_status: :success)
  end
end

This enables us to hide this model from the rest of the system by default until it has reached :success.

Due to system complexity sometimes a request can come for this record while it is in :in_progress state, which will cause IDC to cache a nil value. For us, caching nil is a good thing if the model is in :failed state or if the record is on another shard (good performance characteristics) but not in :in_progress state which means the record is inaccessible until cache_ttl

Proposal

Add another flag that can be persisted to the db for a record called idc_do_not_cache_nil. Internally IDC will load this value and attempt to load from the db. In this scenario there are two possibilities:

1. ActiveRecord returns nil - the default scope couldn't find the record (either :in_progress, :failed or simply not there) - return the nil to the client but do not cache.
2. ActiveRecord returns the record - cache as normal.

The max time idc_do_not_cache_nil is present could be cache_ttl or it could be shorter in our case.

This solves the exact problem we're encountering without introducing locking similar to the Thundering Herd problem: (https://github.com/Shopify/identity_cache/pull/373).

No files or tests are named in the issue. Start by tracing the ActiveRecord integration that handles nil caching and review the proposal's two lookup outcomes; done means the persisted flag prevents caching a nil when the default scope hides the record while preserving normal caching otherwise.