magento / magento/magento2-functional-testing-framework

v4.3.4 - Empty exception messages thrown after test failure

Open
#910 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
PHP
Stars
154
Forks
134
PR merge metrics
No merged PRs in 30d

Description

### Preconditions

1. PHP 8.2
2. Magento 2.4.6
3. MFTF 4.3.4
4. Selenium 3.141.59

### Steps to reproduce

1. Have a test fail

### Expected result

1. A screenshot should be successfully attached to the reports

### Actual result

1. Execution fails with an empty exception message, and all subsequent tests do not run

Example:

```
[clickPlaceOrder] ClickPlaceOrderActionGroup
[waitForPlaceOrderButton] wait for element ".payment-method._active button.action.primary.checkout",30
[saveScreenshot] save screenshot
[] [START AFTER HOOK]
[] [logoutStorefront] StorefrontCustomerLogoutActionGroup
[storefrontSignOut] am on page "customer/account/logout/"
[waitForSignOut] wait for page load 60
[deleteProduct] delete entity "createSimpleProduct","hook"
[deleteCategory] delete entity "createCategory","hook"
[deleteCustomer] delete entity "createCustomer","hook"
[END AFTER HOOK]
ERROR

/******** Beginning execution of AllTestsSuite suite after block ********/
LAST TEST IN SUITE FAILED, TEST AFTER MAY NOT BE SUCCESSFUL

/******** Execution of AllTestsSuite suite after block complete ********/
------------------------------------------------------------

In AllureHelper.php line 29:

[Exception]

Exception trace:
at /var/www/html/vendor/magento/magento2-functional-testing-framework/src/Magento/FunctionalTestingFramework/Allure/AllureHelper.php:29
Magento\FunctionalTestingFramework\Allure\AllureHelper::addAttachmentToCurrentStep() at /var/www/html/vendor/magento/magento2-functional-testing-framework/src/Magento/FunctionalTestingFramework/Extension/TestContextExtension.php:333
Magento\FunctionalTestingFramework\Extension\TestContextExtension->attachExceptionToAllure() at /var/www/html/vendor/magento/magento2-functional-testing-framework/src/Magento/FunctionalTestingFramework/Extension/TestContextExtension.php:142
Magento\FunctionalTestingFramework\Extension\TestContextExtension->testEnd() at /var/www/html/vendor/symfony/event-dispatcher/EventDispatcher.php:220
Symfony\Component\EventDispatcher\EventDispatcher->callListeners() at /var/www/html/vendor/symfony/event-dispatcher/EventDispatcher.php:56
Symfony\Component\EventDispatcher\EventDispatcher->dispatch() at /var/www/html/vendor/codeception/codeception/src/Codeception/Test/Test.php:238
Codeception\Test\Test->realRun() at /var/www/html/vendor/codeception/codeception/src/Codeception/Suite.php:130
Codeception\Suite->run() at /var/www/html/vendor/codeception/codeception/src/Codeception/SuiteManager.php:148
Codeception\SuiteManager->run() at /var/www/html/vendor/codeception/codeception/src/Codeception/Codecept.php:260
Codeception\Codecept->runSuite() at /var/www/html/vendor/codeception/codeception/src/Codeception/Codecept.php:216
Codeception\Codecept->run() at /var/www/html/vendor/codeception/codeception/src/Codeception/Command/Run.php:646
Codeception\Command\Run->runSuites() at /var/www/html/vendor/codeception/codeception/src/Codeception/Command/Run.php:467
Codeception\Command\Run->execute() at /var/www/html/vendor/symfony/console/Command/Command.php:298
Symfony\Component\Console\Command\Command->run() at /var/www/html/vendor/symfony/console/Application.php:1040
Symfony\Component\Console\Application->doRunCommand() at /var/www/html/vendor/symfony/console/Application.php:301
Symfony\Component\Console\Application->doRun() at /var/www/html/vendor/symfony/console/Application.php:171
Symfony\Component\Console\Application->run() at /var/www/html/vendor/codeception/codeception/src/Codeception/Application.php:112
Codeception\Application->run() at /var/www/html/vendor/codeception/codeception/app.php:45
{closure}() at /var/www/html/vendor/codeception/codeception/app.php:46
require() at /var/www/html/vendor/codeception/codeception/codecept:7
include() at /var/www/html/vendor/bin/codecept:119
```

The buggy code is at:

```
public static function addAttachmentToCurrentStep($data, $caption): void
{
if (!is_string($data)) {
try {
$data = serialize($data);
} catch (\Exception $exception) {
throw new \Exception($data->getMessage());
}
}
if (@file_exists($data) && is_file($data)) {
Allure::attachmentFile($caption, $data);
} else {
Allure::attachment($caption, $data);
}
}
```

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with AllureHelper.php at line 29 and trace the call from TestContextExtension::attachExceptionToAllure(). Reproduce a failed test using the listed PHP, Magento, MFTF, and Selenium versions, then verify that the screenshot is attached without an empty exception and that subsequent tests can run.

Written by the indexing model from the issue text.

Assessment

Tech stack
php
Domain
testing-qa
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.