InfluxDB Backend Listener: Limited error information and missing assertion failure details
- Dominant language
- Java
- Stars
- 9.5k
- Forks
- 2.3k
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 5
Description
### Expected behavior
When using the InfluxDB Backend Listener, error metrics should capture complete error information including:
1. Detailed Assertion Failures: Each failed assertion should be reported with:
- Assertion name (e.g., "Response Assertion", "Duration Assertion")
- Specific failure message (e.g., "Expected to contain 'success' but found 'error'")
- Individual error count for each assertion type
2. HTTP Response Failures: Non-assertion failures should report:
- HTTP response codes (e.g., "404", "500")
- Response messages (e.g., "Not Found", "Internal Server Error")
3. Comprehensive Error Categorization: Errors should be properly categorized and distinguishable in InfluxDB/Grafana dashboards
### Actual behavior
The InfluxDB Backend Listener currently provides limited error information:
1. Missing Assertion Details: Assertion failures are completely ignored in error metrics
- No individual assertion failure reporting
- No distinction between different types of assertion failures
- Loss of valuable debugging information
2. Incomplete Error Information: Only HTTP response code/message errors are captured
- Assertion failures are not reflected in error metrics
- No visibility into specific assertion failure reasons
- Difficult to debug test failures without detailed error context
3. Poor Error Debugging: Users cannot identify:
- Which specific assertions are failing
- Why assertions are failing (missing expected text, duration exceeded, etc.)
- Patterns in assertion failures across different test scenarios
> I have the potential solution to resolve this issue: The issue is in the SamplerMetric.add() method in src/components/src/main/java/org/apache/jmeter/visualizers/backend/SamplerMetric.java. The current implementation only creates ErrorMetric objects based on HTTP response codes, ignoring assertion failures entirely.
Required Changes:
- Enhance Error Processing: Modify the error handling logic to process both HTTP response failures and assertion failures
- Add Assertion Failure Support: Create ErrorMetric objects for each failed assertion with assertion name and failure message
- Improve Error Categorization: Distinguish between HTTP response errors and assertion errors in the metrics
### Steps to reproduce the problem
1. Create JMeter Test Plan:
- Add Thread Group
- Add HTTP Request sampler (configure valid URL)
- Add Response Assertion (configure to fail - e.g., check for text that doesn't exist)
- Add InfluxDB Backend Listener
2. Configure Response Assertion:
- Set "Apply to" to "Main sample only"
- Add "Contains" pattern with text that won't be found in response Or add Duration Assertion with very low threshold (e.g., 1ms)
3. Configure InfluxDB Backend Listener:
- Set InfluxDB URL
- Set database name
- Enable "summaryOnly" = false
- Set appropriate measurement name
4. Run Test and observe in InfluxDB/Grafana:
- Expected: Error metrics showing all assertion name (in responseCode) and failure message (in responseMessage) per sample/ transaction
- Actual: No error metrics for assertion failures, only constant Error responseCode (**Assertion Failed**) and single assertion failed message per sample/ transaction are captured
5. Verify Missing Information:
- Check InfluxDB for error metrics with assertion names
- Verify that assertion failure messages are not present
- Confirm that only HTTP response codes appear in error metrics
### JMeter Version
5.6.3
### Java Version
_No response_
### OS Version
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.