Support explicit execution order for --suite, --test, and --task options

Description

While working on resolving pabot issue #306, I encountered a limitation in Robot Framework regarding the execution order of suites and tests when using multiple --suite or --test options. Despite the order of these options in the command line, execution still follows the default alphabetical behavior as described in the Robot Framework User Guide.

For example:
robot --suite "Suite 2" --suite "Suite 1" ./tests
...will still execute Suite 1 before Suite 2 due to the default alphabetical ordering, not the specified order.

This behavior also impacts pabot's --ordering feature when grouping tests into a single subprocess:

{
--suite Tests.Suite 2
--suite Tests.Suite 1
}

Even though the order is explicitly defined, Robot Framework still executes in alphabetical order.

Proposal

Enhance the handling of --suite, --test, and --task options so that their order of appearance in the command line determines the execution order.

This would allow users to control the execution sequence more precisely—especially useful in scenarios where tests have dependencies or need to be run in a specific order (even though this is generally discouraged in test design). It would also benefit pabot and similar tools.

Suggested Traversal Strategy

Adopt a depth-first traversal, combined with the following behavior:

• Folders and files are processed in alphabetical order, with subfolders prioritized before .robot suite files. If needed, prefixes like x__ (where x is a number) can be used to enforce a specific order.
• Tests within a suite retain their written order in the .robot suite file.
• When filtering with patterns like --test "*fail*", matched tests should be stored in a structure that preserves insertion order, such as Python 3.7+ dictionaries.
• As a result, a hypothetical suite tree like the one below will be traversed line by line from top to bottom:

Example Directory Structure (Indented with Test Cases)

tests/
├── performance/
│   └── load_tests.robot
│       ├── Load Test With 1000 Users
│       └── Stress Test Payment API
├── regression/
│   ├── billing/
│   │   ├── invoice_creation.robot
│   │   │   ├── Generate Invoice For Single Product
│   │   │   ├── Generate Invoice With Discount
│   │   │   └── Generate Invoice With Tax Calculation
│   │   └── payment_processing.robot
│   │       ├── Process Credit Card Payment
│   │       └── Handle Payment Failure Gracefully
│   └── user_management/
│       ├── create_user.robot
│       │   ├── Create User With Valid Data
│       │   ├── Create User With Missing Email
│       │   └── Create Admin User
│       └── delete_user.robot
│           ├── Delete Existing User
│           └── Try Delete Nonexistent User
├── smoke/
│   ├── api_healthcheck.robot
│   │   ├── Healthcheck API Returns 200
│   │   └── Status Field Is OK
│   └── login_tests.robot
│       ├── Valid Login Should Succeed
│       ├── Invalid Login Should Fail
│       └── Login Without Password
├── cli_tests.robot
│   ├── CLI Shows Help With --help
│   ├── CLI Fails Gracefully On Invalid Command
│   ├── CLI Executes Known Command Successfully
│   └── Invalid Login Should Fail
├── environment_validation.robot
│   ├── Correct Environment Variables Are Set
│   ├── Database Connection Is Live
│   └── Redis Cache Is Reachable
└── sanity_check.robot
    ├── Application Starts Up
    ├── Main Page Loads Correctly
    └── Basic API Responds With 200

Command Example 1:

robot --test "*fail*" tests

This would result in a dictionary like:

{
    "tests/regression/billing/payment_processing.robot": [
        "Handle Payment Failure Gracefully"
    ],
    "tests/smoke/login_tests.robot": [
        "Invalid Login Should Fail"
    ],
    "tests/cli_tests.robot": [
        "CLI Fails Gracefully On Invalid Command",
        "Invalid Login Should Fail"
    ]
}

Command Example 2:

robot --test "*fail*" --test "*with*" --test "*api*" --test * tests

This would result in a dictionary like:

{
    "tests/regression/billing/payment_processing.robot": [		# first match:
        "Handle Payment Failure Gracefully",				# fail
        "Process Credit Card Payment"					# *
    ],
    "tests/smoke/login_tests.robot": [	
        "Invalid Login Should Fail",					# fail
        "Login Without Password"					# with
        "Valid Login Should Succeed",					# *
    ],
    "tests/cli_tests.robot": [
        "CLI Fails Gracefully On Invalid Command",			# fail
        "Invalid Login Should Fail",					# fail
        "CLI Shows Help With --help",					# with
        "CLI Executes Known Command Successfully"			# *
    ],
    "tests/performance/load_tests.robot:" [
	"Load Test With 1000 Users",					# with
	"Stress Test Payment API"					# api
    ],
    "tests/regression/billing/invoice_creation.robot": [
        "Generate Invoice With Discount",				# with
        "Generate Invoice With Tax Calculation",			# with
        "Generate Invoice For Single Product"				# *
    ],
    "tests/regression/user_management/create_user.robot": [
        "Create User With Valid Data",					# with
        "Create User With Missing Email",				# with
        "Create Admin User"						# *
    ],
    "tests/sanity_check.robot": [
        "Basic API Responds With 200",					# with
        "Application Starts Up",					# *
        "Main Page Loads Correctly"					# *
    ],
    "tests/smoke/api_healthcheck.robot": [
        "Healthcheck API Returns 200",					# api
        "Status Field Is OK"						# *
    ],
    "tests/regression/user_management/delete_user.robot": [
        "Delete Existing User",						# *
        "Try Delete Nonexistent User"					# *
    ],
    "tests/environment_validation.robot": [
        "Correct Environment Variables Are Set",			# *
        "Database Connection Is Live",					# *
        "Redis Cache Is Reachable"					# *
    ]
}

Proposed Evaluation Flow

For each data source (e.g. tests/):

1. Traverse it depth-first.
2. Apply --suite and --rerunfailedsuites options in the order given.
3. Apply --test, --task, and --rerunfailed options in the order given.
4. Apply --include/--exclude tag filters (do not affect ordering, only filtering).
5. Apply --randomize last (if present), which may override manual ordering.

If a --suite option matches a suite path (e.g., *user*) and no --test is provided, all test cases in that suite are included in the declared suite order. Otherwise, only the filtered cases are included.

Compatibility & Versioning

Because this would change the behavior of core options, it would likely require a major version bump.

Summary

This enhancement would:

• Enable users to enforce a specific execution order across suites and tests.
• Improve integration with tools like pabot.
• Better support workflows where order matters (e.g., migrations, performance warmups, or fragile legacy tests).

I’d be happy to refine this proposal further and collaborate on implementation if needed. Feedback welcome!

Start by reproducing the command-line examples with repeated --suite and --test options, then trace the option handling and suite traversal entry points in Robot Framework. Done means declared option order is preserved for suites, tests, and tasks while tag filtering and --randomize retain the proposed behavior; the issue names no specific files or tests.