project-chip / project-chip/certification-tool

[Bug] TC-IDM-10.5 Adding extra clusters to a standard device type will cause the test to fail

Open
#933 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Bug Needs Triage
Dominant language
Shell
Stars
82
Forks
47
Avg merge
1d 15h
Merged PRs (30d)
17

Description

Describe the bug

When we add an extra onoff cluster to a device type that does not originally contain an onoff cluster, it causes test case TC-IDM-10.5 to fail. According to the test plan description, PIXIT.allow_extra_clusters can be added to allow this case to pass when extra clusters exist. However, when I communicated with the ATL engineer, they believed that this type of bypass flag cannot be used in product certification. But there is no relevant prohibition described in either the Spec or the test plan. Is there an accurate description regarding this point? How should this be understood specifically?

Steps to reproduce the behavior
  1. Add onoff cluster in a Thermostat device
  2. When use default command run TC-IDM-10.5 case, it will failed
  3. When add param --bool-arg fail_on_extra_clusters:false, it will pass
Expected behavior

We can use --bool-arg fail_on_extra_clusters:false param to pass the TC-IDM-10.5 case with extra cluster in a DUT

Log files

No response

PICS file

No response

Screenshots
Image Image
Environment

No response

Additional Information

Related issue: https://github.com/project-chip/certification-tool/issues/577

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the implementation and test definition for TC-IDM-10.5, then trace how --bool-arg fail_on_extra_clusters:false is handled. Compare that behavior with the cited test plan and specification, and review related issue #577. Done means the certification interpretation is resolved and the tool behavior or documentation is updated accordingly.

Written by the indexing model from the issue text.

Assessment

Domain
devtools, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.