ruby / ruby/psych

YAML 1.2 compliance possibly breaking VCR

Open
#648 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Ruby
Stars
597
Forks
223
Avg merge
11h 23m
Merged PRs (30d)
3

Description

This is an issue branching off from https://github.com/ruby/psych/pull/613#issuecomment-1709239243.

We're experimenting migrating a Rails app we have to latest JRuby (9.4.3.0), an app that has some large VCR fixtures.

Initially we hit the the code point limit issue that https://github.com/ruby/psych/pull/613 lets us bypass, but after increasing the code point limit that the above PR let's us configure, we came across parsing errors like this one:

the leading empty lines contain more spaces (8) than the first non-empty line. while scanning a block scalar at line 32385 column 5
     # ./config/initializers/psych.rb:6:in `parse_stream'

(the initializer monkey patch is a hack while we can't get https://github.com/ruby/psych/pull/647 to work on our end)

In this case, the fixture had several empty newlines in a multiline field, using a block scalar header like field: |+2.

After deleting that field from the yaml fixture altogether as part of troubleshooting, all fixtures get parsed and we're able to run Rspec specs again using VCR. But some of the specs that were previously passing now fail, with Webmock not recognizing the URIs for the requests recorded in the fixtures.

We suspect this might be happening due to YAML 1.2 not liking how the URIs are escaped, and we're still validating this. @headius asked me to create this issue so it can be tracked together with https://github.com/ruby/psych/issues/642, which might also be related to YAML 1.2 compliance (see also https://github.com/jruby/jruby/pull/7600#issuecomment-1712771885)

I'll update this issue as we find out more about it, ideally with a test comparing the same use case with YAML 1.1 and YAML 1.2.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the parsing failure with the VCR fixture's block scalar header and the URI fixture that WebMock does not recognize. Compare the same cases under YAML 1.1 and YAML 1.2, using the linked Psych and JRuby pull requests and issues for context. Done means the behavior is explained and covered by a focused regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, ruby, yaml
Domain
devtools, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.