google / google/testrun

Cater for scenarios when device is no longer responding on resolved IP address

Open
#1,365 3 comments 0 reactions 0 assignees View on GitHub
BE triaged
Dominant language
Python
Stars
56
Forks
20
Avg merge
15h 54m
Merged PRs (30d)
12

Description

**Describe the bug**
Take a look at the report [here](https://github.com/user-attachments/files/22519005/report.pdf).

The device in question requested an IP address during Startup, but then during Monitoring it abandoned its DHCP-allocated IP address, and reverted to using a completely different static IP address instead. This lasted for the remainder of the Monitoring phase, and then throughout all subsequent tests.

I'm not suggesting that this behaviour specifically should be captured/reported by Testrun, but the resulting report features a complete list of false negatives and false positives, for example the below suggesting that no HTTP server was found. It's not as simple as no HTTP server was found per se, the device wasn't even there on that IP address to begin with, and this is misleading.

Image

Similar false negatives in the Services section of the report too, as elsewhere.

Image

**Expected behavior**
You'll see throughout testrun.log that Testrun is very much aware that `Resolved device IP: None`, which is symptomatic of the above issue, and so there should be a mechanism in place to 'invalidate' or 'null' those relevant test modules or sections of the report, to prevent false negatives or false positives from being portrayed.

Image

**Error logs**
[report (69).zip](https://github.com/user-attachments/files/22519278/report.69.zip)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.