commoncriteria / commoncriteria/PSD
PSD module definitions of FDP_APC_EXT.1 and FDP_PDC_EXT.1 contain tests that have excessive amounts of activities within them
Nobody has claimed this yet.
- Dominant language
- HTML
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
There are many cases in the modules of PSD in which one test within FDP_APC_EXT.1 covers a handful of distinct activities and expected results, which leads to unnecessary ambiguity and confusion in testing and test feedback. The test activities for this SFR should be refactored to encompass a single test activity and expected result like in other PPs and PP-Modules. Sub-tests can be used where a common setup is applied across multiple test activities, rather than the current approach, which uses a lengthy sequence of test steps for setup and multiple sets of observations relying on that setup. Nearly all test activities defined across all of the modules can benefit from this refactoring, both for legibility and for automation efforts.
For the purposes of illustration, test FDP_APC_EXT.1:1 in MOD_KM_V2.0 could be split out into multiple tests to increase readability like so, though note that this is applicable to nearly all tests in all of the definitions of FDP_APC_EXT.1 and FDP_PDC_EXT.1, not just the ones in the KM module:
Test FDP_APC_EXT.1:1: [conditional] If “switching can be initiated only through express user action” is selected in FDP_SWI_EXT.1.1 in the PSD PP", the evaluator shall configure the TOE and the Operational Environment in accordance with the operational guidance. The evaluator shall then connect an authorized peripheral device for each peripheral device type selected in FDP_PDC_EXT.3.1/KM, and run an instance of a text editor on each connected computer. Finally, the evaluator shall connect a display to each computer in order to see all computers at the same time and turn on the TOE. The evaluator shall then perform the following tests, entering text or moving the cursor to verify which connected computer is selected after each attempted switching action:
- Test FDP_APC_EXT.1:1.1:
For each switching method selected in FDP_SWI_EXT.2.2, the evaluator shall switch selected computers in accordance with the operational user guidance, and verify that it succeeds.
- Test FDP_APC_EXT.1:1.2:
For each peripheral device type selected in FDP_PDC_EXT.3.1/KM, the evaluator shall attempt to switch the device to more than one computer at once and verify that the TOE ignores all such commands or otherwise prevents the operation from executing.
- Test FDP_APC_EXT.1:1.3: [conditional- if "keyboard" is selected in FDP_PDC_EXT.3.1/KM]
The evaluator shall attempt to control the computer selection using the following standard keyboard shortcuts, where ‘#’ represents a computer channel number, and verify that the selected computer is not switched:
- Control ‐ Control ‐ # ‐ Enter
- Shift ‐ Shift ‐ #
- Num Lock ‐ Minus ‐ #
- Scroll Lock ‐ Scroll Lock ‐ #
- Scroll Lock ‐ Scroll Lock ‐ Function #
- Scroll Lock ‐ Scroll Lock ‐ arrow (up or down)
- Scroll Lock ‐ Scroll Lock ‐ # ‐ enter
- Control ‐ Shift ‐ Alt ‐ # ‐ Enter
- Alt ‐ Control ‐ Shift ‐ #
- Test FDP_APC_EXT.1:1.4: [conditional - if "mouse" is selected in FDP_PDC_EXT.3.1/KM]
The evaluator shall attempt to switch to other connected computers using the pointing device and verify that it does not succeed.
- Test FDP_APC_EXT.1:1.5: [conditional - if "peripheral devices using a guard" is selected in FDP_SWI_EXT.2.2]
The evaluator shall attempt to switch to other connected computers using the peripheral device and guard by only performing some of the steps outlined in the operational user guidance, and verify that it does not succeed.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the PSD module definitions for FDP_APC_EXT.1 and FDP_PDC_EXT.1, including MOD_KM_V2.0 and its FDP_APC_EXT.1:1 example. Compare the existing test structures with other PPs and PP-Modules. Done means refactoring the applicable tests into single activities with single expected results, using sub-tests only for shared setup.
Written by the indexing model from the issue text.
Assessment
- Domain
- documentation, testing-qa
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100