apache / apache/jmeter

InfluxDB Backend Listener: Limited error information and missing assertion failure details

Open
#6,491 0 comments 0 reactions 0 assignees View on GitHub
defect to-triage
Dominant language
Java
Stars
9.5k
Forks
2.3k
Avg merge
1d 22h
Merged PRs (30d)
5

Description

### Expected behavior

When using the InfluxDB Backend Listener, error metrics should capture complete error information including:

1. Detailed Assertion Failures: Each failed assertion should be reported with:
- Assertion name (e.g., "Response Assertion", "Duration Assertion")
- Specific failure message (e.g., "Expected to contain 'success' but found 'error'")
- Individual error count for each assertion type
2. HTTP Response Failures: Non-assertion failures should report:
- HTTP response codes (e.g., "404", "500")
- Response messages (e.g., "Not Found", "Internal Server Error")
3. Comprehensive Error Categorization: Errors should be properly categorized and distinguishable in InfluxDB/Grafana dashboards

### Actual behavior

The InfluxDB Backend Listener currently provides limited error information:

1. Missing Assertion Details: Assertion failures are completely ignored in error metrics
- No individual assertion failure reporting
- No distinction between different types of assertion failures
- Loss of valuable debugging information
2. Incomplete Error Information: Only HTTP response code/message errors are captured
- Assertion failures are not reflected in error metrics
- No visibility into specific assertion failure reasons
- Difficult to debug test failures without detailed error context
3. Poor Error Debugging: Users cannot identify:
- Which specific assertions are failing
- Why assertions are failing (missing expected text, duration exceeded, etc.)
- Patterns in assertion failures across different test scenarios

> I have the potential solution to resolve this issue: The issue is in the SamplerMetric.add() method in src/components/src/main/java/org/apache/jmeter/visualizers/backend/SamplerMetric.java. The current implementation only creates ErrorMetric objects based on HTTP response codes, ignoring assertion failures entirely.

Required Changes:
- Enhance Error Processing: Modify the error handling logic to process both HTTP response failures and assertion failures
- Add Assertion Failure Support: Create ErrorMetric objects for each failed assertion with assertion name and failure message
- Improve Error Categorization: Distinguish between HTTP response errors and assertion errors in the metrics

### Steps to reproduce the problem

1. Create JMeter Test Plan:
- Add Thread Group
- Add HTTP Request sampler (configure valid URL)
- Add Response Assertion (configure to fail - e.g., check for text that doesn't exist)
- Add InfluxDB Backend Listener
2. Configure Response Assertion:
- Set "Apply to" to "Main sample only"
- Add "Contains" pattern with text that won't be found in response Or add Duration Assertion with very low threshold (e.g., 1ms)
3. Configure InfluxDB Backend Listener:
- Set InfluxDB URL
- Set database name
- Enable "summaryOnly" = false
- Set appropriate measurement name
4. Run Test and observe in InfluxDB/Grafana:
- Expected: Error metrics showing all assertion name (in responseCode) and failure message (in responseMessage) per sample/ transaction
- Actual: No error metrics for assertion failures, only constant Error responseCode (**Assertion Failed**) and single assertion failed message per sample/ transaction are captured
5. Verify Missing Information:
- Check InfluxDB for error metrics with assertion names
- Verify that assertion failure messages are not present
- Confirm that only HTTP response codes appear in error metrics

### JMeter Version

5.6.3

### Java Version

_No response_

### OS Version

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.