google / google/googletest

[FR]: Making `EXPECT_THAT` of `(Unordered)ElementsAreArray` human/AI readable

Open
#4,934 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
C++
Stars
39.6k
Forks
10.9k
Avg merge
6d 13h
Merged PRs (30d)
1

Description

Does the feature exist in the most recent commit?

I guess yes (version that is used in chromium).

Why do we need this feature?

For daily chromium development.

Describe the proposal.

Motivation

I think that the current output of EXPECT_THAT of (Unordered)ElementsAreArray is not human/AI readable. We can improve it.

Example

Code: Something like https://source.chromium.org/chromium/chromium/src/+/main:content/browser/preloading/prerender/prerender_browsertest.cc;l=8357;drc=02e3f72738b5ad02e8f30ab40ea1467550a91f2c

Result:

../../content/browser/preloading/prerender/prerender_browsertest.cc:8988: Failure
Value of: prerender_entries
Expected: has 4 elements where
element #0 is equal to Entry{source=65 AccurateTriggering=1 Eligibility=1 FailureReason=0 HoldbackStatus=1 PreloadingPredictor=50 PreloadingType=4 ReadyTime=1247 SpeculationEagerness=2 TimeToNextNavigation=1247 TriggeringOutcome=5 },
element #1 is equal to Entry{source=65 AccurateTriggering=0 Eligibility=1 FailureReason=0 HoldbackStatus=1 PreloadingPredictor=50 PreloadingType=4 ReadyTime=1247 SpeculationEagerness=2 TimeToNextNavigation=1247 TriggeringOutcome=4 },
element #2 is equal to Entry{source=65 AccurateTriggering=0 Eligibility=1 FailureReason=186 HoldbackStatus=1 PreloadingPredictor=50 PreloadingType=4 SpeculationEagerness=2 TimeToNextNavigation=1247 TriggeringOutcome=6 },
element #3 is equal to Entry{source=65 AccurateTriggering=0 Eligibility=1 FailureReason=0 HoldbackStatus=1 PreloadingPredictor=50 PreloadingType=4 SpeculationEagerness=2 TimeToNextNavigation=1247 TriggeringOutcome=3 }
  Actual: { Entry{source=65 AccurateTriggering=1 Eligibility=1 FailureReason=0 HoldbackStatus=1 PreloadingPredictor=50 PreloadingType=4 ReadyTime=1247 SpeculationEagerness=2 TimeToNextNavigation=1247 TriggeringOutcome=5 }, Entry{source=65 AccurateTriggering=0 Eligibility=1 FailureReason=0 HoldbackStatus=1 PreloadingPredictor=50 PreloadingType=4 SpeculationEagerness=2 TimeToNextNavigation=1247 TriggeringOutcome=3 }, Entry{source=65 AccurateTriggering=0 Eligibility=1 FailureReason=0 HoldbackStatus=1 PreloadingPredictor=50 PreloadingType=4 SpeculationEagerness=2 TimeToNextNavigation=1247 TriggeringOutcome=9 }, Entry{source=65 AccurateTriggering=0 Eligibility=1 FailureReason=0 HoldbackStatus=1 PreloadingPredictor=50 PreloadingType=4 SpeculationEagerness=2 TimeToNextNavigation=1247 TriggeringOutcome=9 } }, whose element #1 (Entry{source=65 AccurateTriggering=0 Eligibility=1 FailureReason=0 HoldbackStatus=1 PreloadingPredictor=50 PreloadingType=4 SpeculationEagerness=2 TimeToNextNavigation=1247 TriggeringOutcome=3 }) isn't equal to Entry{source=65 AccurateTriggering=0 Eligibility=1 FailureReason=0 HoldbackStatus=1 PreloadingPredictor=50 PreloadingType=4 ReadyTime=1247 SpeculationEagerness=2 TimeToNextNavigation=1247 TriggeringOutcome=4 }

It's not readable for me. What I'm currently doing is:

  1. Find "whose element #1".
  2. Copy the line.
  3. Paste to an editor.
  4. Edit
  5. Eye diff

Editted result:

whose element #1 (
Entry{source=65 AccurateTriggering=0 Eligibility=1 FailureReason=0 HoldbackStatus=1 PreloadingPredictor=50 PreloadingType=4 SpeculationEagerness=2 TimeToNextNavigation=1247 TriggeringOutcome=3 }
) isn't equal to 
Entry{source=65 AccurateTriggering=0 Eligibility=1 FailureReason=0 HoldbackStatus=1 PreloadingPredictor=50 PreloadingType=4 ReadyTime=1247 SpeculationEagerness=2 TimeToNextNavigation=1247 TriggeringOutcome=4 }

Then, I got Diff: prerender_entries[1], ReadyTime=1247, TriggeringOutcome=3/4.

Sometimes I ask to an AI to visualize the diff.

This work is in most-inner loop of iteration. It's OK for oneshot, but I need to do it for every edit and compile.

Possible solutions

I prefer

  • Human/AI readable output.
  • Structural output rather than natural english.

Imporved output:

../../content/browser/preloading/prerender/prerender_browsertest.cc:8988: Failure
Value of: prerender_entries
Expected: With `ElementsAreArray`
  - Entry{source=65 AccurateTriggering=1 Eligibility=1 FailureReason=0 HoldbackStatus=1 PreloadingPredictor=50 PreloadingType=4 ReadyTime=1247 SpeculationEagerness=2 TimeToNextNavigation=1247 TriggeringOutcome=5 },
  - Entry{source=65 AccurateTriggering=0 Eligibility=1 FailureReason=0 HoldbackStatus=1 PreloadingPredictor=50 PreloadingType=4 ReadyTime=1247 SpeculationEagerness=2 TimeToNextNavigation=1247 TriggeringOutcome=4 },
  - Entry{source=65 AccurateTriggering=0 Eligibility=1 FailureReason=186 HoldbackStatus=1 PreloadingPredictor=50 PreloadingType=4 SpeculationEagerness=2 TimeToNextNavigation=1247 TriggeringOutcome=6 },
  - Entry{source=65 AccurateTriggering=0 Eligibility=1 FailureReason=0 HoldbackStatus=1 PreloadingPredictor=50 PreloadingType=4 SpeculationEagerness=2 TimeToNextNavigation=1247 TriggeringOutcome=3 }
Actual:
  - Entry{source=65 AccurateTriggering=1 Eligibility=1 FailureReason=0 HoldbackStatus=1 PreloadingPredictor=50 PreloadingType=4 ReadyTime=1247 SpeculationEagerness=2 TimeToNextNavigation=1247 TriggeringOutcome=5 },
  - Entry{source=65 AccurateTriggering=0 Eligibility=1 FailureReason=0 HoldbackStatus=1 PreloadingPredictor=50 PreloadingType=4 SpeculationEagerness=2 TimeToNextNavigation=1247 TriggeringOutcome=3 },
  - Entry{source=65 AccurateTriggering=0 Eligibility=1 FailureReason=0 HoldbackStatus=1 PreloadingPredictor=50 PreloadingType=4 SpeculationEagerness=2 TimeToNextNavigation=1247 TriggeringOutcome=9 },
  - Entry{source=65 AccurateTriggering=0 Eligibility=1 FailureReason=0 HoldbackStatus=1 PreloadingPredictor=50 PreloadingType=4 SpeculationEagerness=2 TimeToNextNavigation=1247 TriggeringOutcome=9 } }
Note that the expected element #1 isn't equal to the actual one.
  - Expected #1: Entry{source=65 AccurateTriggering=0 Eligibility=1 FailureReason=0 HoldbackStatus=1 PreloadingPredictor=50 PreloadingType=4 SpeculationEagerness=2 TimeToNextNavigation=1247 TriggeringOutcome=3 }
  - Actual   #1: Entry{source=65 AccurateTriggering=0 Eligibility=1 FailureReason=0 HoldbackStatus=1 PreloadingPredictor=50 PreloadingType=4 ReadyTime=1247 SpeculationEagerness=2 TimeToNextNavigation=1247 TriggeringOutcome=4 }

Points:

  1. Comparison using LHS/RHS.
  2. Same indentation and format for LHS/RHS.
  3. Say the conclusion first and avoid embedding parameters. (Note that the expected element #1 isn'st equal to the actual one.\n - {lhs}\n - {rhs} rather than Actual {actual}, whose element #1 ({rhs}) isn't equal to {lhs}. Note also that the natural English version reverts the order as actual/expected, which is avoided in gtest.)

WDYT?

Is the feature specific to an operating system, compiler, or build system version?

No.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the googletest implementation and tests that format failures for EXPECT_THAT with ElementsAreArray and UnorderedElementsAreArray. Compare the current output with the proposed structural LHS/RHS format, then update coverage so ordered and unordered mismatches report the differing elements consistently. Done means the relevant matcher failures are readable and the existing tests pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.