ARMmbed / ARMmbed/greentea-client

Role of greentea-client in controlling test flow not clear

Open
#27 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
3
Forks
13
PR merge metrics
No merged PRs in 30d

Description

The definite roles of greentea-client include
* performing synchronisation/handshake (i.e. `GREENTEA_SETUP`) with the host
* communicating with the host via key-value pairs

It's not clear whether greentea-client or utest should be responsible for controlling test case flow (i.e. starting tests, reporting results, etc.). For example,
* greentea-client provides `void GREENTEA_TESTCASE_START(const char *test_case_name)` for starting a test.
* But utest calls `greentea_send_kv(TEST_ENV_TESTCASE_START, source->get_description())` to directly send a KV pair containing the "start" signal, without using what greentea-client provides.

Fixing this is _not_ as simple as switching utest to call greentea-client's test control API. The current greentea-client API doesn't provide everything needed by utest, for example the former does not offer a function to report test summary (passes & failures).

We may need to improve the interface of greentea-client in either of the two ways:
* Remove functions for controlling test flow from greentea-client, keeping only handshake and KV capabilities. Callers (e.g. utest) set keys and values.
* Extend functions for controlling test flow, and refactor utest to use those functions. Make keys for test flow internal to greentea-client.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by comparing the greentea-client test-control APIs with utest's direct greentea_send_kv calls, including GREENTEA_SETUP and GREENTEA_TESTCASE_START. Determine which component should own test flow and what summary reporting requires. Done means the interface decision is documented and the affected APIs and callers have a consistent design.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
embedded-iot, testing-qa
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.