Shopify / Shopify/ruby-lsp

Ruby LSP Test Runner - Windows Path Normalization Bug

Open
#3,881 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

help-wanted pinned
Dominant language
Ruby
Stars
2k
Forks
281
Avg merge
2h 14m
Merged PRs (30d)
6

Description

On Windows, the Test Explorer runs tests successfully but never receives results - the spinner keeps spinning indefinitely and test status never updates. This is caused by a path normalization mismatch when looking up the reporter port from the temp database.

Environment
  • OS: Windows 10/11
  • Ruby: 3.4.0 (via RubyInstaller)
  • ruby-lsp: 0.26.4
  • ruby-lsp-rspec: 0.1.28
  • VS Code: Latest
  • Ruby LSP Extension: Latest
Root Cause

The LSP reporter stores paths in test_reporter_port_db.json with Windows-style backslashes and lowercase drive letters:

{
  "d:\\source\\repos\\myproject": 12345
}

However, Ruby's Dir.pwd returns forward slashes with uppercase drive letters:

Dir.pwd
# => "D:/source/repos/myproject"

When lsp_reporter.rb looks up the port using Dir.pwd directly, the hash lookup fails due to this mismatch. The code then silently falls back to a no-op StringIO, so tests execute but results are never sent back to VS Code.

Steps to Reproduce
  1. Open a Ruby/Rails project on Windows in VS Code
  2. Ensure ruby-lsp and ruby-lsp-rspec are in your Gemfile
  3. Open a spec file
  4. Click "Run" on a test via CodeLens or Test Explorer
  5. Observe: Tests run in terminal, but Test Explorer spinner never stops and results aren't shown
Expected Behavior

Test results should be reported back to VS Code Test Explorer, showing pass/fail status for each test.

Actual Behavior
  • Tests execute successfully (visible in terminal output)
  • Test Explorer spinner keeps spinning indefinitely
  • No pass/fail indicators appear
  • No error messages are shown (silent failure)
Suggested Fix

Normalize paths before the hash lookup in the LSP reporter. Something like:

# Before lookup, normalize Dir.pwd to match database format
def normalized_working_directory
  if Gem.win_platform?
    Dir.pwd.gsub('/', '\\').sub(/^([A-Z]):/) { $1.downcase + ':' }
  else
    Dir.pwd
  end
end

Or alternatively, normalize both the stored paths and lookup paths to a canonical format (e.g., forward slashes, lowercase on Windows).

Workaround

Add this to the top of spec/spec_helper.rb (before any other requires):

# Workaround for ruby-lsp Windows path mismatch
# See: https://github.com/Shopify/ruby-lsp/issues/XXXX
if ENV['RUBY_LSP_TEST_RUNNER'] && !ENV['RUBY_LSP_REPORTER_PORT']
  require 'tmpdir'
  require 'json'
  port_db_path = File.join(Dir.tmpdir, 'ruby-lsp', 'test_reporter_port_db.json')
  if File.exist?(port_db_path)
    db = JSON.load_file(port_db_path)
    # Normalize current directory to match database format (backslashes, lowercase drive)
    normalized_pwd = Dir.pwd.gsub('/', '\\').sub(/^([A-Z]):/) { Regexp.last_match(1).downcase + ':' }
    ENV['RUBY_LSP_REPORTER_PORT'] = db[normalized_pwd].to_s if db[normalized_pwd]
  end
end

This manually sets the RUBY_LSP_REPORTER_PORT environment variable with the correct port by doing the path normalization that the LSP reporter should be doing.

Additional Context

This issue makes the VS Code Test Explorer integration completely non-functional on Windows. The silent fallback to StringIO makes debugging very difficult - there are no error messages indicating why results aren't being reported.

The fix should be straightforward since it's purely a path string normalization issue. Happy to submit a PR if pointed to the right location in the codebase.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating lsp_reporter.rb and the code that reads test_reporter_port_db.json for the reporter port. Reproduce the lookup on Windows with the differing path formats, then verify that the reporter sends test results to VS Code instead of falling back to StringIO. Done means Test Explorer displays pass/fail results and the existing non-Windows behavior remains intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
ruby
Domain
devtools, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.