How to use robot-server and pybot with not fiddling with env variables?

Start with the robot-server and pybot entry points, the fixture that stores the randomly assigned port, and the related issue #99. Compare the documented environment-variable workflow with the proposed default behavior, then determine which approach should be adopted. Done means the separate server and test commands have a documented, reliable way to use the same port.

After #51 the robot server starts on a random port and that port is stored on the fixture.

Related issue:

• https://github.com/plone/plone.app.robotframework/issues/99

While this is very good for normal testing this breaks the workflow of people that were using the robot-server script to run the server and the pybot script to run the tests.

The reason is that the two scripts are run separately and there is no simple way for the pybot script to know which port the robot server is using.

We can solve this issue in many way:

Force everybody to set the relevant environment variables

For example we can change the documentation to suggest this way of running the server:

ZSERVER_PORT=55001 ./bin/robot-server plone.app.robotframework.testing.PLONE_ROBOT_TESTING 

and this way to run pybot:

ZSERVER_HOST=localhost ZSERVER_PORT=55001 ./bin/pybot -t '*' src/plone/app/robotframework/tests/test_autologin_library.robot

Pro:

• We do not touch plone.testing, no action needed by the testing team

Con:

• It is not really friendly and KISS compliant
• We need to update the documentation

Restore the previous defaults unless some variable is set to force the random ports.

We could yet another environment variable and change the default for the port from 55001 to 0 only if ZSERVER_USE_RANDOM_PORT is truish.
Or we can just ZSERVER_PORT=0.

Pro:

• We restore a well documented behavior

Con:

• We need to change again plone.testing
• Maybe yet another variable is needed (there are already many)
• The jenkins job needs to take care of setting that variable before running the tests

I personally prefer this second approach because it seems more natural to me.