magento / magento/magento2-functional-testing-framework
v4.3.4 - Empty exception messages thrown after test failure
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 154
- Forks
- 134
- PR merge metrics
- No merged PRs in 30d
Description
### Preconditions
1. PHP 8.2
2. Magento 2.4.6
3. MFTF 4.3.4
4. Selenium 3.141.59
### Steps to reproduce
1. Have a test fail
### Expected result
1. A screenshot should be successfully attached to the reports
### Actual result
1. Execution fails with an empty exception message, and all subsequent tests do not run
Example:
```
[clickPlaceOrder] ClickPlaceOrderActionGroup
[waitForPlaceOrderButton] wait for element ".payment-method._active button.action.primary.checkout",30
[saveScreenshot] save screenshot
[] [START AFTER HOOK]
[] [logoutStorefront] StorefrontCustomerLogoutActionGroup
[storefrontSignOut] am on page "customer/account/logout/"
[waitForSignOut] wait for page load 60
[deleteProduct] delete entity "createSimpleProduct","hook"
[deleteCategory] delete entity "createCategory","hook"
[deleteCustomer] delete entity "createCustomer","hook"
[END AFTER HOOK]
ERROR
/******** Beginning execution of AllTestsSuite suite after block ********/
LAST TEST IN SUITE FAILED, TEST AFTER MAY NOT BE SUCCESSFUL
/******** Execution of AllTestsSuite suite after block complete ********/
------------------------------------------------------------
In AllureHelper.php line 29:
[Exception]
Exception trace:
at /var/www/html/vendor/magento/magento2-functional-testing-framework/src/Magento/FunctionalTestingFramework/Allure/AllureHelper.php:29
Magento\FunctionalTestingFramework\Allure\AllureHelper::addAttachmentToCurrentStep() at /var/www/html/vendor/magento/magento2-functional-testing-framework/src/Magento/FunctionalTestingFramework/Extension/TestContextExtension.php:333
Magento\FunctionalTestingFramework\Extension\TestContextExtension->attachExceptionToAllure() at /var/www/html/vendor/magento/magento2-functional-testing-framework/src/Magento/FunctionalTestingFramework/Extension/TestContextExtension.php:142
Magento\FunctionalTestingFramework\Extension\TestContextExtension->testEnd() at /var/www/html/vendor/symfony/event-dispatcher/EventDispatcher.php:220
Symfony\Component\EventDispatcher\EventDispatcher->callListeners() at /var/www/html/vendor/symfony/event-dispatcher/EventDispatcher.php:56
Symfony\Component\EventDispatcher\EventDispatcher->dispatch() at /var/www/html/vendor/codeception/codeception/src/Codeception/Test/Test.php:238
Codeception\Test\Test->realRun() at /var/www/html/vendor/codeception/codeception/src/Codeception/Suite.php:130
Codeception\Suite->run() at /var/www/html/vendor/codeception/codeception/src/Codeception/SuiteManager.php:148
Codeception\SuiteManager->run() at /var/www/html/vendor/codeception/codeception/src/Codeception/Codecept.php:260
Codeception\Codecept->runSuite() at /var/www/html/vendor/codeception/codeception/src/Codeception/Codecept.php:216
Codeception\Codecept->run() at /var/www/html/vendor/codeception/codeception/src/Codeception/Command/Run.php:646
Codeception\Command\Run->runSuites() at /var/www/html/vendor/codeception/codeception/src/Codeception/Command/Run.php:467
Codeception\Command\Run->execute() at /var/www/html/vendor/symfony/console/Command/Command.php:298
Symfony\Component\Console\Command\Command->run() at /var/www/html/vendor/symfony/console/Application.php:1040
Symfony\Component\Console\Application->doRunCommand() at /var/www/html/vendor/symfony/console/Application.php:301
Symfony\Component\Console\Application->doRun() at /var/www/html/vendor/symfony/console/Application.php:171
Symfony\Component\Console\Application->run() at /var/www/html/vendor/codeception/codeception/src/Codeception/Application.php:112
Codeception\Application->run() at /var/www/html/vendor/codeception/codeception/app.php:45
{closure}() at /var/www/html/vendor/codeception/codeception/app.php:46
require() at /var/www/html/vendor/codeception/codeception/codecept:7
include() at /var/www/html/vendor/bin/codecept:119
```
The buggy code is at:
```
public static function addAttachmentToCurrentStep($data, $caption): void
{
if (!is_string($data)) {
try {
$data = serialize($data);
} catch (\Exception $exception) {
throw new \Exception($data->getMessage());
}
}
if (@file_exists($data) && is_file($data)) {
Allure::attachmentFile($caption, $data);
} else {
Allure::attachment($caption, $data);
}
}
```
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with AllureHelper.php at line 29 and trace the call from TestContextExtension::attachExceptionToAllure(). Reproduce a failed test using the listed PHP, Magento, MFTF, and Selenium versions, then verify that the screenshot is attached without an empty exception and that subsequent tests can run.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100