PAIR-code / PAIR-code/deliberate-lab

bug: Booting participants does not propagate — booted users keep going / stay "waiting"

Open
#1,180 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
96
Forks
40
Avg merge
2d 11h
Merged PRs (30d)
20

Description

Summary

Experimenters report that booting participants has no effect on the
participant's screen
. Feld reports:

Booting seems to not propagate to participants — booted several for
inactivity, they still are waiting.

Some participants, after being booted, were able to proceed on their
local system. Today, one participant appeared to have been booted for
failing an attention check in the Intro stage, but complained that they
were stuck on the Transfer stage.

Code review points to a concrete, reproducible root cause for the
attention-check case, plus two secondary hypotheses for the more general
"still waiting" case.

Primary hypothesis (high confidence): booting during an attention check is a dead-end

When an experimenter boots a participant, the backend does not always set
BOOTED_OUT
. If the participant is currently in ATTENTION_CHECK status,
the boot is instead downgraded to ATTENTION_TIMEOUT:

// functions/src/participant.endpoints.ts (bootParticipant)
if (participant.currentStatus === ParticipantStatus.ATTENTION_CHECK) {
  participant.currentStatus = ParticipantStatus.ATTENTION_TIMEOUT;
} else {
  participant.currentStatus = ParticipantStatus.BOOTED_OUT;
}

The problem: the participant frontend has no terminal UI for
ATTENTION_TIMEOUT.
The participant view only renders blocking overlays
for three statuses, and ATTENTION_TIMEOUT is not one of them:

  • renderBootedPopup() → renders only when status is BOOTED_OUT
  • renderAttentionPopup() → renders only when status is ATTENTION_CHECK
  • renderTransferPopup() → renders only when status is TRANSFER_PENDING

So the sequence is:

  1. Participant is in ATTENTION_CHECK → the "Are you still there?" overlay
    is showing (this overlay blocks the UI).
  2. Experimenter clicks Boot.
  3. Backend sets status to ATTENTION_TIMEOUT.
  4. On the participant's browser, the attention-check overlay disappears
    (status is no longer ATTENTION_CHECK), and no replacement overlay is
    rendered
    (status is not BOOTED_OUT).
  5. The participant is dropped back onto the underlying live stage view and
    can keep interacting — exactly matching "able to proceed on their
    local system"
    and "booted for failing an attention check … stuck on
    the Transfer stage."

There is also no client-side redirect for this case. routeToEndExperiment
handles an ATTENTION_TIMEOUT redirect code, but it is never invoked in
reaction to a server-driven status change — it only runs on the booted
popup's "Exit experiment" button (which never appears here), on transfer
failure, and on success.

Note the dashboard does not prevent this: the Boot button is only
disabled when isParticipantEndedExperiment(...) is true, which is false
for ATTENTION_CHECK — so booting a mid-attention-check participant is
allowed.

Why this likely explains BOTH reports

The attention check is a manual experimenter action used precisely against
inactive / unresponsive participants ("Are you still there?"). So the
population an experimenter boots "for inactivity" heavily overlaps with the
population currently sitting in ATTENTION_CHECK. Booting them yields
ATTENTION_TIMEOUT → no overlay, no redirect → they remain on their stage
and appear to "still be waiting" / keep going.

Relevant code
  • functions/src/participant.endpoints.ts (L678–713) — bootParticipant
  • frontend/src/components/participant_view/participant_view.ts (L135–208) — renderPopups / renderBootedPopup / renderAttentionPopup
  • utils/src/participant.ts (L96–120) — ParticipantStatus enum (ATTENTION_TIMEOUT = 'ATTENTION_FAIL')
  • frontend/src/services/participant.service.ts (L471–497) — routeToEndExperiment (never triggered by a status change)
  • frontend/src/components/experiment_dashboard/participant_summary.ts (L291–320) — Boot button not disabled for ATTENTION_CHECK

Secondary hypothesis A: no client-side reaction to terminal status changes

Even for the plain BOOTED_OUT path, termination is passive: the app
relies on the <booted-popup> overlay rendering and the participant
manually clicking Exit experiment to be redirected. There is no MobX
reaction/observer that fires routeToEndExperiment (or otherwise locks
the UI) when profile.currentStatus transitions to a terminal state
server-side. If the overlay fails to cover a particular stage view, or the
participant ignores it, no automatic redirect/lock occurs.

  • frontend/src/services/participant.service.ts (L263–305) — profile subscription sets currentStageViewId on every snapshot but does not react to a terminal status

Secondary hypothesis B: booted-while-disconnected never receives the update

Participants booted "for inactivity" are, by definition, likely
disconnected (tab backgrounded/closed, network dropped). The status change
only reaches the browser through the live Firestore onSnapshot listener on
their participant doc. If the listener isn't active at boot time, the
overlay won't appear until they reconnect and the service re-subscribes —
and if they never reconnect, the experimenter dashboard may continue to show
them as present/"waiting". This also does not remove them from wait-stage
counts / turn order for the rest of the cohort, which may be part of what
the experimenter perceives as "still waiting."

Proposed steps to reproduce

Repro A — Boot during an attention check (primary; expected to fail today)
  1. Run locally: ./run_locally.sh (frontend at http://localhost:4201).
  2. Create/open an experiment with at least one interactive stage after the
    intro (a Transfer or Chat stage matches the report well).
  3. Open the experiment as a participant in one browser (Browser P) and
    start the experiment so they're on a stage.
  4. As the experimenter in another browser (Browser E), open the cohort
    dashboard and click the warning / attention-check icon for that
    participant.
  5. Confirm Browser P now shows the "Are you still there?" overlay
    (status = ATTENTION_CHECK). Do not click "Yes, continue".
  6. In Browser E, click the Boot (block) icon for that participant and
    confirm.
  7. Observe (bug): In Browser P the attention overlay disappears and
    no "You have been removed" popup appears; the participant can
    continue interacting with the stage and is not redirected. Backend
    status is ATTENTION_TIMEOUT (ATTENTION_FAIL), not BOOTED_OUT.
  8. Expected: the participant should be shown a terminal message and/or
    redirected (to Prolific with attentionFailRedirectCode when Prolific
    integration is enabled) and blocked from proceeding.
Repro B — Plain boot, no manual exit (secondary A)
  1. Same setup, but do not send an attention check first (participant
    status = IN_PROGRESS).
  2. Boot the participant from the dashboard.
  3. Observe: Browser P shows the <booted-popup> overlay, but the
    participant is only redirected if they click Exit experiment. Confirm
    there is no automatic redirect/lock; verify whether the overlay reliably
    covers every stage type (e.g. wait stage, transfer stage, chat).
Repro C — Boot a disconnected participant (secondary B)
  1. Same setup; get a participant onto a wait stage.
  2. In Browser P, simulate inactivity/disconnect (background the tab, or go
    offline via DevTools → Network → Offline).
  3. Boot the participant from the dashboard.
  4. Bring Browser P back online / foreground.
  5. Observe: whether the terminal state is applied on reconnect, and
    whether the dashboard/wait-counts for the rest of the cohort update to
    exclude the booted participant.

Open questions

  • Is downgrading a boot to ATTENTION_TIMEOUT intentional, or should an
    explicit boot always yield BOOTED_OUT?
  • Should the participant frontend treat all terminal statuses
    (BOOTED_OUT, ATTENTION_TIMEOUT, transfer failures) uniformly — a
    blocking terminal overlay plus automatic routeToEndExperiment?
  • Should there be a client-side reaction that redirects/locks as soon as
    currentStatus becomes terminal, rather than depending on a manual
    "Exit experiment" click?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with ./run_locally.sh and Repro A, then inspect functions/src/participant.endpoints.ts, frontend/src/components/participant_view/participant_view.ts, utils/src/participant.ts, and frontend/src/services/participant.service.ts. Compare the attention-check and plain-boot status transitions, including the dashboard code in participant_summary.ts. Done means the reported boot scenarios produce a blocking terminal outcome and the relevant local reproduction checks pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
firebase, typescript
Domain
backend, frontend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
64/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.