paritytech / paritytech/host-rust-core

Two iOS suites report green without running what they cover

Open
#816 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

host-work
Dominant language
Swift
Stars
10
Forks
3
Avg merge
1d 10h
Merged PRs (30d)
148

Description

Two suites report green in ios-pr.yml without having run the code they exist
to cover. Both are conditions in the app tree rather than anything CI does
wrong, but the consequence lands here, because this is where that tree is now
compiled and tested.

A skipped suite over code that is not otherwise covered

hosts/ios/polkadot-app/polkadot-app.xctestplan lists ClaimAssetServiceTests
under skippedTests for the CoinageTests target. The service it covers
computes claim amounts and submits the chain calls acting on them.

In the test job 232 suites run and this one is absent, while the other new
suites under Packages/Coinage/Tests/IncomingPayment all run. So the job is
green over that code having executed none of it.

A suite that asserts the opposite of what it guards

TRUAPI_RUNTIME_DEFAULT is set only in Configs/base.nightly.xcconfig.
ios-pr.yml builds and tests DevCI, which does not set it, so
ProductRuntimeSettingsTests compiles to its #else branch and asserts
#expect(!InMemorySettingsManager().isTrUAPIRuntimeEnabled).

The suite passes. What it asserts is the inverse of the behaviour the flag
introduces, so the default has never been exercised in a configuration where it
applies.

Where each fix belongs

The second is ours. Which configuration the test job builds is decided in
ios-pr.yml, so covering the flagged branch is a CI change and touches nothing
in the imported tree.

The first is not. Unskipping means editing a file inside hosts/ios, which
becomes an adaptation that every future refresh has to carry and reconcile, and
scripts/refresh-host-import.sh now measures exactly that cost. Worth raising
against the app repository instead, so the suite runs on both sides.

Found while reviewing #774.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with ios-pr.yml and trace which configuration the test job builds, then compare it with Configs/base.nightly.xcconfig and the branches exercised by ProductRuntimeSettingsTests. Done means the PR test job executes the flagged runtime branch and still reports the expected result. The skipped ClaimAssetServiceTests suite is in hosts/ios and is identified for follow-up in the app repository, not this change.

Written by the indexing model from the issue text.

Assessment

Tech stack
swift
Domain
ci-cd, mobile-dev, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
64/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.