project-chip / project-chip/matter-test-scripts

[TC-BINFO-2.1] V1.4.2 YAML script is failing in step 2 and 62 while running the test against V1.3/V1.2 sample app as DUT

Open
#600 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
19
Forks
7
Avg merge
3d 17h
Merged PRs (30d)
3

Description

Description:

When executing TC-BINFO-2.1 using DUTs built with 1.3 or 1.2 Matter SDK, the test fails at:

  • Step 2
  • Step 62

Reason:

The current 1.4.2 YAML script validates responses strictly as per 1.4.2 expected values, without accepting valid responses from previous versions (1.3/1.2). However, the test plan allows those earlier values as valid.

Expected Behavior:

The script should accept any response values mentioned in the test plan, including those defined for earlier SDK releases (1.3/1.2), to support backward compatibility.

Suggested Fix:

  • Update the YAML script for TC-BINFO-2.1 to:
  • Accept multiple valid values where applicable, as defined in the test plan.
  • Support backward compatibility for 1.3/1.2 DUTs.

Additional Notes:

Test Plan already includes acceptable values for all releases. This logic needs to be reflected in the YAML validation.

Test Plan Snapshot:

Image

Image

Steps to Reproduce:

  • Build the TH version: v2.13-beta4+summer2025.
  • Upload the attached PICS XML file to the TH.
  • Select the TC-BINFO-2.1 test case.
  • Build the All-Clusters-app using v1.2-branch/v1.3-branch.
  • Advertice the DUT using: ./chip-all-clusters-app
  • Start the test case in TH.

PICS XML File: Basic Information Cluster Test Plan.zip

Failure logs for reference: BINFO_2_1_2025_06_03_14_42_43.log

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the YAML definition for TC-BINFO-2.1 and inspect the validations for steps 2 and 62. Compare those checks with the acceptable 1.2 and 1.3 values in the test-plan snapshot, then use the linked failure log and reproduction steps to verify that both older DUT versions pass while current expected values remain accepted.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, yaml
Domain
testing-qa
Issue type
Bug
Difficulty
2/5
Estimated time
Half a day
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.