SWG needs to consider the nuts and bolt of the testability of the API

The SWG needs to discuss and consider the testability of the requirements of the specification. Specifically, in order to test certain requirements of the specification (e.g. jobs conformance class) the processes being tested needs to have certain characteristics. For example, the process has to run asynchronously for a sufficient time to allow the test engine to test all the intermediate status messages.

So the question is, how do we convey, describe these preconditions for testability?

With regards to the jobs conformance class, some ideas discussed during the TB17 CITE kickoff included:

1. Require that the server being tested implement an "Echo" process. This process could include a pause argument to simulate long running times for job creating and execution.
2. Have the client submit an execute request to any process it deploys. The test engine can parse that execute request and figure out if, for example, the process is being run asynchronously and then decide to test the intermediate status messages.
   In all this discussion, it was emphasised that the test suite is NOT testing the output of the process. That is irrelevant. Only the interface behaviour should be tested.

Although I have concentrated on the jobs conformance class in the description of this issue, the issue is a general one and the SWG needs to consider in the ATS how all the requirement might actually be tested in the real world.