marklogic / marklogic/cloud-enablement-aws

instance-state in the node-state table (DynamoDB/SimpleDB) is a hardcoded literal, never reflects real instance status

Open
#268 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Xonsh
Stars
20
Forks
19
Avg merge
15h 19m
Merged PRs (30d)
6

Description

mdb-update-state() in mlcmd/scripts/mdb-functions.xsh hardcodes instance-state to the literal string "running":

mdb-update state $1 state-instance $MARKLOGIC_INSTANCE instance-state running instance-time <[ fn:current-dateTime() ]>
Per the comment in ddb-functions.xsh line 315 ("Keep quiet this is run every minute from cron"), this field is refreshed periodically regardless of actual node health.

Observed impact: after scaling an ASG-managed cluster to 0 (full stop), the DynamoDB node-state item kept reporting instance-state=running with instance-time still advancing for several minutes past InService=0, then froze permanently at "running" with a stale instance ID once the instance was actually terminated. Anything consuming this table to infer real node liveness (automation, dashboards, other cluster tooling) will be misled -- there is no way to distinguish "genuinely running" from "was running as of the last cron tick, possibly hours or days ago."

Suggested fix: either rename the field to something that does not imply liveness (e.g. last-seen-state), or have it reflect an actual check (e.g. query the instance's real EC2/process status) rather than a hardcoded literal.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in mlcmd/scripts/mdb-functions.xsh at mdb-update-state(), then read the related comment around line 315 of ddb-functions.xsh and trace how instance-state is consumed. The issue needs an agreed replacement semantics—renaming the field or checking actual EC2/process status—before implementation can be considered done.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws
Domain
cloud, databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.