rspec / rspec/rspec

`start_with` and `end_with` matcher failure output does not include a diff on multi-line strings

Open
#141 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Ruby
Stars
117
Forks
51
Avg merge
59m
Merged PRs (30d)
3

Description

Subject of the issue

RSpec generally has great failure output, but when using start_with or end_with on a multi-line string, the output is quite hard to read. RSpec is totally capable of better output here, though; simply by changing expect(multline_str).to start_with(expected) to expect(multiline.lines.first(n).join).to eq(expected) the output becomes much more readable because it includes a diff.

Similarly, expect(multiline_str).to end_with(expected) is quite hard to read, but expect(multiline_str.lines.last(n).join).to eq(expected) is much easier to read because it includes a diff.

Can start_with and end_with be improved to automatically diff the first or last n lines if they are dealing with multiline strings?

Your environment
  • Ruby version: 3.2.2
  • rspec-expectations version: 3.12.0
Steps to reproduce

Put this in tmp/start_and_end_with_spec.rb:

module MyApp
  module SomeModule
    RSpec.describe "RSpec matchers" do
      describe "start_with" do
        it "fails on a multi-line string in a way that's hard to read" do
          expect(::File.read(__FILE__)).to start_with(<<~EOS)
            module MyApp
              module SomeModule
                RSpec.describe "RSpec matchers" do
                  describe "start_with", :a_change do
                    it "fails on a multi-line string in a way that's hard to read" do
                      expect(::File.read(__FILE__)).to start_with(<<~EOS)
          EOS
        end
      end

      describe "end_with" do
        it "fails on a multi-line string in a way that's hard to read" do
          expect(::File.read(__FILE__)).to end_with(<<~EOS)
                    end
                  end
                end # a change
              end
            end
          EOS
        end
      end

      describe "eq", :improved do
        it "can be used in place of `start_with` for more readable failure output" do
          expect(::File.read(__FILE__).lines.first(6).join).to eq(<<~EOS)
            module MyApp
              module SomeModule
                RSpec.describe "RSpec matchers" do
                  describe "start_with", :a_change do
                    it "fails on a multi-line string in a way that's hard to read" do
                      expect(::File.read(__FILE__)).to start_with(<<~EOS)
          EOS
        end

        it "can be used in place of `end_with` for more readable failure output" do
          expect(::File.read(__FILE__).lines.last(5).join).to eq(<<~EOS)
                    end
                  end
                end # a change
              end
            end
          EOS
        end
      end
    end
  end
end

Run it to reproduce. Run rspec tmp/start_and_end_with_spec.rb --tag "~improved" to see the current output for start_with and end_with and rspec tmp/start_and_end_with_spec.rb --tag improved for the improved output I'd like to see from these matchers instead.

Expected behavior

rspec tmp/start_and_end_with_spec.rb --tag improve produces output close to what I'd expect:

Run options: include {:improved=>true}
FF

Failures:

  1) RSpec matchers eq can be used in place of `start_with` for more readable failure output
     Failure/Error:
                 expect(::File.read(__FILE__).lines.first(6).join).to eq(<<~EOS)
                   module MyApp
                     module SomeModule
                       RSpec.describe "RSpec matchers" do
                         describe "start_with", :a_change do
                           it "fails on a multi-line string in a way that's hard to read" do
                             expect(::File.read(__FILE__)).to start_with(<<~EOS)
                 EOS

       expected: "module MyApp\n  module SomeModule\n    RSpec.describe \"RSpec matchers\" do\n      describe \"start_... in a way that's hard to read\" do\n          expect(::File.read(__FILE__)).to start_with(<<~EOS)\n"
            got: "module MyApp\n  module SomeModule\n    RSpec.describe \"RSpec matchers\" do\n      describe \"start_... in a way that's hard to read\" do\n          expect(::File.read(__FILE__)).to start_with(<<~EOS)\n"

       (compared using ==)

       Diff:
       @@ -1,7 +1,7 @@
        module MyApp
          module SomeModule
            RSpec.describe "RSpec matchers" do
       -      describe "start_with", :a_change do
       +      describe "start_with" do
                it "fails on a multi-line string in a way that's hard to read" do
                  expect(::File.read(__FILE__)).to start_with(<<~EOS)

     # ./tmp/start_and_end_with_spec.rb:31:in `block (3 levels) in <module:SomeModule>'

  2) RSpec matchers eq can be used in place of `end_with` for more readable failure output
     Failure/Error:
                 expect(::File.read(__FILE__).lines.last(5).join).to eq(<<~EOS)
                           end
                         end
                       end # a change
                     end
                   end
                 EOS

       expected: "        end\n      end\n    end # a change\n  end\nend\n"
            got: "        end\n      end\n    end\n  end\nend\n"

       (compared using ==)

       Diff:
       @@ -1,6 +1,6 @@
                end
              end
       -    end # a change
       +    end
          end
        end

     # ./tmp/start_and_end_with_spec.rb:42:in `block (3 levels) in <module:SomeModule>'

Finished in 0.01014 seconds (files took 0.0588 seconds to load)
2 examples, 2 failures

Failed examples:

rspec ./tmp/start_and_end_with_spec.rb:30 # RSpec matchers eq can be used in place of `start_with` for more readable failure output
rspec ./tmp/start_and_end_with_spec.rb:41 # RSpec matchers eq can be used in place of `end_with` for more readable failure output
Actual behavior

rspec tmp/start_and_end_with_spec.rb --tag "~improve" produces the output I find hard to read:

Run options: exclude {:improved=>true}
FF

Failures:

  1) RSpec matchers start_with fails on a multi-line string in a way that's hard to read
     Failure/Error:
                 expect(::File.read(__FILE__)).to start_with(<<~EOS)
                   module MyApp
                     module SomeModule
                       RSpec.describe "RSpec matchers" do
                         describe "start_with", :a_change do
                           it "fails on a multi-line string in a way that's hard to read" do
                             expect(::File.read(__FILE__)).to start_with(<<~EOS)
                 EOS

       expected "module MyApp\n  module SomeModule\n    RSpec.describe \"RSpec matchers\" do\n      describe \"start_...e\n              end\n            end\n          EOS\n        end\n      end\n    end\n  end\nend\n" to start with "module MyApp\n  module SomeModule\n    RSpec.describe \"RSpec matchers\" do\n      describe \"start_... in a way that's hard to read\" do\n          expect(::File.read(__FILE__)).to start_with(<<~EOS)\n"
     # ./tmp/start_and_end_with_spec.rb:6:in `block (3 levels) in <module:SomeModule>'

  2) RSpec matchers end_with fails on a multi-line string in a way that's hard to read
     Failure/Error:
                 expect(::File.read(__FILE__)).to end_with(<<~EOS)
                           end
                         end
                       end # a change
                     end
                   end
                 EOS

       expected "module MyApp\n  module SomeModule\n    RSpec.describe \"RSpec matchers\" do\n      describe \"start_...e\n              end\n            end\n          EOS\n        end\n      end\n    end\n  end\nend\n" to end with "        end\n      end\n    end # a change\n  end\nend\n"
     # ./tmp/start_and_end_with_spec.rb:19:in `block (3 levels) in <module:SomeModule>'

Finished in 0.01098 seconds (files took 0.05936 seconds to load)
2 examples, 2 failures

Failed examples:

rspec ./tmp/start_and_end_with_spec.rb:5 # RSpec matchers start_with fails on a multi-line string in a way that's hard to read
rspec ./tmp/start_and_end_with_spec.rb:18 # RSpec matchers end_with fails on a multi-line string in a way that's hard to read

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reproducer in tmp/start_and_end_with_spec.rb and run the two commands provided for the current and improved output. Trace the start_with and end_with matcher failure paths, then add coverage showing multiline failures include a useful diff while preserving existing behavior. Done means the matcher failures resemble the expected diff output.

Written by the indexing model from the issue text.

Assessment

Tech stack
ruby
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.