project-chip / project-chip/certification-tool

[Bug] TC-DA-1.7 fails if run in GUI but passes running direct from Docker

Open
#507 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Awaiting Dev Feedback Bug SDK v1.4
Dominant language
Shell
Stars
83
Forks
47
Avg merge
1d 15h
Merged PRs (30d)
17

Description

Describe the bug

The test fails indicating 2 DUT are required even though these parameters are passed to the starting python script as shown in the log. Adding the two passphrases and discriminators (from the project config) does not work. However running the test directly from docker does.

INFO       | 2024-12-04 13:46:46.666764 | Running Python Test: TC_DA_1_7
INFO       | 2024-12-04 13:47:36.626356 | Sending command to SDK container: python3 /root/python_testing/scripts/sdk/test_harness_client.py sdk/TC_DA_1_7 TC_DA_1_7 --tests test_TC_DA_1_7 --trace-to json:log --discriminator 3947 3945 --passcode 20240118 20240116 --wifi-ssid CERT --wifi-passphrase 12345678 --paa-trust-store-path /paa-root-certs  --PICS /var/tmp/pics

 ' File "/root/python_testing/scripts/sdk/TC_DA_1_7.py", line 119, in test_TC_DA_1_7
    asserts.assert_equal(len(self.matter_test_config.discriminators), 2, "This test requires 2 DUTs")`
Steps to reproduce the behavior

Boot the TH GUI, select TC-DA-1.7 (this requires two DUT with differing passcode/discriminator/DACs) and a correct PAA to be installed on /paa-root-certs as per the test plan.

Correctly set the GUI project configurations, note the two discriminators : "3947 3945", and setup_codes "20240118 20240116", used in the configurations,

"dut_config": {
"discriminator": "3947 3945",
"setup_code": "20240118 20240116",
"pairing_mode": "ble-wifi",
"chip_timeout": null,
"chip_use_paa_certs": true,
"trace_log": true
},
"test_parameters": {
"wifi-ssid": "CERT",
"wifi-passphrase": "12345678",
"paa-trust-store-path": "/paa-root-certs"
}

Run the test.

Expected behavior

The test should run as expected.

If this test is run directly from the docker it passes

docker run -v /var/paa-root-certs:/paa-root-certs -v /var/run/dbus/system_bus_socket:/var/run/dbus/system_bus_socket -v $(pwd):/launch_dir --privileged --network host -it connectedhomeip/chip-cert-bins:50bcad170e6def6d56f2f5b34522a82aea9f3190
python3 python_testing/TC_DA_1_7.py --commissioning-method ble-wifi --discriminator 3947 3945 --passcode 20240118 20240116 --storage-path admin_storage.json --paa-trust-store-path /paa-root-certs --wifi-ssid CERT --wifi-passphrase 12345678 --ble-interface-id 0
Log files

No response

PICS file

No response

Screenshots

No response

Environment

Version: 2.10.2+spring2024 Sha: 2a350c8

Additional Information

No response

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with python_testing/TC_DA_1_7.py and scripts/sdk/test_harness_client.py, then compare the GUI-generated command and project configuration with the direct Docker invocation. Trace how the two discriminators and passcodes are parsed and passed to the test. Done means TC-DA-1.7 runs successfully from the GUI with both DUT configurations.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
testing-qa, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
62/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.