User deletion leaves stale cache references: GraphQL queries fail site-wide with NoSuchUserException (unguarded loadUserById in addVersionProperties; HostCache not invalidated on delete)

Problem Statement

Deleting a dotCMS user correctly rewrites all database references (mod_user, locked_by, …) to the
replacement user — but cached Host/Contentlet objects on running nodes keep the deleted user's id. Any
GraphQL query whose result window touches one of those stale cached objects then fails entirely with
Internal Server Error(s) while executing query, because contentlet hydration performs an unguarded
user lookup.

This caused a production incident: a routine backoffice bulk-deletion of 25 users was followed
immediately by ~72 minutes of site-wide GraphQL search failures (thousands of errors/min) until a full
cache flush cleared the stale references. No restart, reindex, or data fix was needed — the DB was
already correct; the dangling references existed only in memory.

Mechanism (verified on main):

1. UserFactoryImpl.loadUserById (UserFactoryImpl.java:115) throws NoSuchUserException(userId) when
   the row is gone — with the bare user id as the message (log signature:
   ERROR datafetcher.SiteFieldDataFetcher - user-<uuid>).
2. GraphQL hydration (graphQLDataFetchOptions → VERSION_INFO) routes into
   DefaultTransformStrategy.addVersionProperties, where line 377
   (loadUserById(contentlet.getModUser())) and line 390 (loadUserById(lockedBy)) are
   unguarded — one missing user row fails the whole query. The tolerant pattern already exists in
   the same class: lines 160–161 wrap the identical lookups in Try.of(...).getOrNull().
3. DefaultGraphQLErrorHandler converts the field failure into a top-level error with
   data.search = null, so the client loses the entire payload, not one field.

Why the stale references survive deletion:

• ESContentFactoryImpl.updateUserReferences → reindexReplacedUserContent
  (ESContentFactoryImpl.java:1708) decides what to cache-evict/reindex via an Elasticsearch query
  (+modUser:<oldId>). Anything the index doesn't return is never evicted, so any live/working index
  divergence leaves stale cache entries behind.
• UserAPIImpl.delete (UserAPIImpl.java:505) performs no HostCache invalidation at all;
  HostAPIImpl.find serves Host objects straight from hostCache, so cached sites keep the
  pre-deletion modUser indefinitely — which made the GraphQL host field resolver
  (/search[N]/host) the dominant failure point.

Steps to Reproduce

1. Warm the cache: run a GraphQL search returning content/site objects last modified by user X.
2. Delete user X from the backoffice (DB references are rewritten to a replacement user — this works).
3. Re-run the same GraphQL search without flushing caches.
4. The query fails with {"errors":[{"message":"Internal Server Error(s) while executing query"}],"data":{"search":null}}; logs show NoSuchUserException: <userId of X> from addVersionProperties. The REST Content API returns the same content cleanly (different transform path).
5. Flush all caches → the same query succeeds.

Acceptance Criteria

1. addVersionProperties tolerates a missing user: apply the existing Try.of(...).getOrNull()
   pattern (lines 160–161) to the lookups at lines 377 and 390 — a deleted user yields a null field,
   never a failed query. This alone downgrades this incident class from site-wide outage to a null field.
2. User deletion invalidates affected caches deterministically: clear HostCache in the delete path, and
   don't rely solely on an index query to enumerate contentlet-cache evictions.
3. Immediately after a user deletion, GraphQL search over content that referenced them returns data
   with no errors and no manual cache flush.

dotCMS Version

Observed on Evergreen 26.07.21-01 (multi-node). Code paths verified present on current main.

External Links

• Customer ticket: https://dotcms.freshdesk.com/a/tickets/38889
• Nearest prior art: #18256 (site variables failing on a non-existing user — same family, different surface)

Start in DefaultTransformStrategy.addVersionProperties and compare the user lookups at lines 377 and 390 with the tolerant pattern at lines 160–161. Then trace UserAPIImpl.delete, HostAPIImpl.find, and ESContentFactoryImpl.updateUserReferences/reindexReplacedUserContent to understand cache invalidation paths. Done means deletion leaves no stale references that make GraphQL searches fail, without requiring a manual cache flush.