Network settings config upgrade in start script is brittle

Open
#907 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Refactor
Clarity
Mostly clear
Activity status
Quiet
Tech stack
shell
Domain
devops

Research direction

Read the upgrade_soroban_config function in start around lines 393-437 and the network-upgrade flow around lines 672-718, then inspect the linked failure and related issues #906 and #555. Reproduce the upgrade path and trace how get-settings-upgrade-txs output, curl submission, and metrics polling interact. Done means the flow confirms each intended transaction and no longer depends on global counters, fixed line counts, or piped stdout parsing.

Written by the indexing model from the issue text.

Description

feature request
What problem does your feature solve?

The network settings config upgrade logic in the start script is brittle. The upgrade_soroban_config function uses stellar-core get-settings-upgrade-txs to generate transactions, submits them via curl to core's HTTP endpoint, and confirms they were applied by polling the global ledger.transaction.count metric.

For example:

https://github.com/stellar/quickstart/blob/6357b286e43c856c68b2b6b7690f8a520d1b7f5d/start#L672-L718

The script reads transactions and transaction IDs from stdout line-by-line, submits each via curl, then waits for the global transaction count metric to increment:

while [ "`curl -s http://localhost:11626/metrics | jq -r '.metrics."ledger.transaction.count".count'`" != "$TX_COUNT" ]; do sleep 1; done

This is brittle in several ways:

  • Transaction confirmation by global counter: It does not verify that the specific transaction succeeded, only that the total transaction count increased. If any other transaction occurs, or if a transaction fails but is still counted, the logic breaks.
  • Output format coupling: The script checks if [ $line_count = 9 ] vs 7 lines to detect whether a restore operation is included in the output, coupling it tightly to the exact output format of stellar-core get-settings-upgrade-txs which can change between versions.
  • Pipe-based parsing of stdout: The entire flow reads tx blobs and tx IDs via read from a piped subshell, which is fragile and hard to debug when something goes wrong.

@sisuresh and I have noticed some recent flaky build failures that may be related to this brittleness:

Related: #906, #555

What would you like to see?

Replace the brittle shell-based transaction submission and confirmation logic with something more robust. This could be part of a small Rust CLI tool (#906) that handles transaction submission and confirmation directly, or another approach that avoids relying on polling global metrics and parsing stdout line counts.

What alternatives are there?
  • Improve the shell script: Add retries, check transaction results directly via the /tx endpoint response, and make the output parsing more resilient. This improves reliability but still leaves the fundamental brittleness of doing this in bash.
  • Use stellar-cli: Ship stellar-cli with quickstart and use it for transaction submission. Downside is that stellar-cli is further downstream and harder to keep in sync with unreleased stellar-core changes.
  • Build into a small Rust CLI: As proposed in #906, a minimal Rust tool could handle this logic more robustly with proper error handling and transaction result checking.
Dominant language
Shell
Stars
222
Forks
236
Avg merge
1d 12h
Merged PRs (30d)
12

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from stellar/quickstart

All issues in stellar/quickstart

Similar issues

More Shell/Bash issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.