An interrupted first boot silently bricks the instance: starter import leaves site permissions unwritten and every Apps API call 403s forever

Problem Statement

An interrupted first boot leaves a dotCMS instance permanently unusable, and neither the restart nor the logs tell the user. Every Apps API call returns 403 forever. The only recovery is destroying the volumes and starting over.

This was found while diagnosing #37262 (the @dotcms/create-app CLI failure). It is the backend half of that issue and is filed separately because it is broader than the CLI: any interrupted first boot hits it, whatever started the container.

Steps to Reproduce

Using docker/docker-compose-examples/single-node-demo-site/docker-compose.yml:

1. docker compose up -d from clean volumes.
2. ~25s in — during the starter import, while ImportStarterUtil is processing com.dotmarketing.beans.Tree — kill the dotCMS container: docker kill <dotcms>.
3. Start it again: docker start <dotcms>. It boots and reports a clean startup.
4. Issue an API token as admin@dotcms.com, resolve the default site, then call the Apps API for that site.

Expected Behavior

Either the instance repairs itself on restart (re-running the incomplete import), or it fails loudly — refusing to serve, logging a clear "incomplete initialization" error, and telling the operator to recreate the instance.

Actual Behavior

Startup looks completely normal:

14:05:24  INFO  db.DotCMSInitDb - There are no inodes - initializing db with starter site
14:05:24  INFO  db.DotCMSInitDb - STARTER_DATA_LOAD property is set to ...zip, loading starter data
14:05:50  INFO  startup.Catalina - Server startup in [37136] milliseconds

But the admin user has no permissions on the site that was supposedly imported:

DotSecurityException: User 'Admin User [ID: dotcms.org.1][email:admin@dotcms.com]'
  does not have READ permissions on Site 'demo.dotcms.com'

Every Apps API call 403s, and it never clears:

T+39s  /api/v1/appconfiguration        -> 200
T+41s  POST /api/v1/authentication/api-token -> 200
T+41s  GET  /api/v1/site/defaultSite         -> 200
T+41s  GET/POST /api/v1/apps/dotema-config-v2/{siteId} -> 403
   … 193 consecutive attempts over ~7 minutes …
T+440s GET/POST /api/v1/apps/dotema-config-v2/{siteId} -> 403

193 attempts, zero successes. Waiting does not help. A clean boot of the same stack returns 200 for all three calls at 46s.

Analysis

The interrupted import left the site's permission rows unwritten. On restart DotCMSInitDb still reported "There are no inodes" and re-ran Task00004LoadStarter, but the permissions never appeared — so the re-run did not restore the state the first run left half-written.

Three things seem wrong, in increasing order of severity:

1. The re-run does not repair. Task00004LoadStarter runs again but the resulting instance is still missing permissions.
2. The failure is silent. Startup logs report success. Nothing indicates the instance is unusable.
3. The symptom is misleading. A missing-permissions data problem surfaces to every client as 403 Forbidden, which reads as an auth or licensing problem. In #37262 this cost significant investigation time — the original hypothesis blamed user.isAdmin() swallowing an exception via Try.of(…).getOrElse(false), which turned out to be wrong: there is no exception, the data is simply absent.

A partially-imported starter arguably should not be committed at all — if the import is not atomic, an interruption leaves a state the system cannot distinguish from a valid one.

Impact

High for first-run experience. Anyone whose first boot is interrupted — a crash, a laptop sleeping, Ctrl-C, an OOM kill, a container racing its database — gets an instance that looks healthy and is permanently broken, with no diagnostic pointing at the cause. This is the mechanism behind the user-visible failure in #37262.

Environment

dotcms/dotcms:latest · single-node-demo-site compose · starter 20260630 · macOS + OrbStack, dotCMS constrained to 2 CPUs / 4G.

Evidence limits: one host, one starter, one image, kill point fixed at 25s. Which kill-points corrupt and which are survivable is unmapped.

Related

• #37262 — @dotcms/create-app local Docker failure; this is its backend half

Start with DotCMSInitDb and Task00004LoadStarter, then trace the starter import through ImportStarterUtil. Reproduce the interrupted import using docker/docker-compose-examples/single-node-demo-site/docker-compose.yml and check whether site permission rows survive a restart. Done means an interrupted first boot either recovers correctly or reports a clear incomplete-initialization failure instead of serving persistent 403 responses.