forcedotcom / forcedotcom/code-analyzer

[BUG][code-analyzer] 5.16.0 silently drops custom PMD rulesets that point at a sibling file by relative path

Open
#2,087 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
240
Forks
52
Avg merge
1d 23h
Merged PRs (30d)
5

Description

### Have you tried to resolve this issue yourself first?

- [x] I confirm I have gone through the above steps and still have an issue to report.

### Bug Description

After upgrading to `@salesforce/plugin-code-analyzer` 5.16.0, a scan that used to run our custom PMD rules stops running them, and nothing fails. The exit code is still 0. The only sign is one Critical violation attached to no file, with a summary of `Found 1 violation(s) across 0 file(s)`. A CI gate that does not use `--severity-threshold` reads green while zero Apex rules ran.

`sf code-analyzer config` and `sf code-analyzer rules` show the cause: `Cannot resolve rule/ruleset reference 'scanner/full.xml/MyRule'`.

**Why it happens.** Our repo has two custom rulesets: a full one that defines the rules, and a smaller one that reuses some of them with ``, a path relative to the repo root. Up to 5.15.x, PMD resolved that path against the repo being scanned, because the PMD process was started from there.

code-analyzer-core PR #503 (the CWE-427 hardening) now starts the PMD process from the engine's own install folder (`cwd: __dirname`). The PR calls this behaviour-preserving because every argument handed to `java` is an absolute path. That is true for the paths the engine passes, but not for paths written inside a custom ruleset: PMD still resolves those from the process's working directory, which is now the wrong folder. The whole ruleset then fails to load, and the PMD engine drops out of the run.

The 5.16.0 release notes do not mention it.

Originally filed on the core repo as forcedotcom/code-analyzer-core#508. Cross-filing here because that repo's issue bot reports its config as incomplete, so the report never reached triage.

### Output / Logs

```
› Error: Code Analyzer [8:43:51.703]:
› Failed to get rules from engine with name 'pmd' due to the following
› error:
› PMD errored when attempting to load a custom ruleset
› "scanner/full.xml/MyRule". Make sure the
› resource is a valid ruleset file on disk or on the Java classpath.

› PMD Exception:
› | Cannot load ruleset
› scanner/full.xml/MyRule: Cannot resolve
› rule/ruleset reference
› 'scanner/full.xml/MyRule'. Make sure the
› resource is a valid file or URL and is on the CLASSPATH. Use --debug (or a
› fine log level) to see the current classpath.

› If you wish to ignore this error and disable this engine, then update
› your Code Analyzer configuration with:
› engines:
› pmd:
› disable_engine: true

=== Summary

Found 1 violation(s) across 0 file(s):
1 Critical severity violation(s) found.
```

The command exits 0.

### Steps To Reproduce

1. `scanner/full.xml` defines `MyRule`. `scanner/subset.xml` contains ``.
2. `code-analyzer.yml` in the repo root lists both files under `engines.pmd.custom_rulesets`.
3. Run `sf code-analyzer rules --rule-selector pmd:MyRule` from the repo root.

On 5.15.x the rule is listed. On 5.16.0 the ruleset fails to load with the error above. The same configuration loaded cleanly on 5.14.0 and 5.15.0.

### Expected Behavior

Either of these:

- Paths inside a custom ruleset resolve against the config file's folder or the workspace root, as they did before 5.16.0. One way: keep the working directory pinned, but add the workspace root to PMD's classpath, so `scanner/...` resolves as a classpath resource.
- Or the `custom_rulesets` docs and the 5.16.0 release notes state that a custom ruleset must be self-contained (no relative `` to another file).

In both cases, a ruleset that fails to load should fail the run, not exit 0 with the engine silently removed.

### Operating System

macOS 26.6.2 (build 25G83), Apple Silicon

### Salesforce CLI Version

@salesforce/cli/2.150.6 darwin-arm64 node-v24.19.0

### Code Analyzer Plugin (code-analyzer) Version

code-analyzer 5.16.0 (code-analyzer-core 0.53.0, code-analyzer-pmd-engine 0.46.0, PMD 7.26.0)

### Node Version

v22.22.3

### Java Version

openjdk version "26.0.2.1" 2026-08-18

### Python Version

_No response_

### Additional Context (Screenshots, Files, etc)

The bundled PMD is 7.26.0 on both 5.15.0 and 5.16.0, so this is not a PMD change. The only relevant difference between the two plugin releases is the working directory of the spawned `java` process.

### Workaround

Generate the smaller ruleset as a self-contained copy of the rules it needs, instead of pointing at the full one. That is what we now do, guarded by a test that rejects any file-path `` in our rulesets.

### Urgency

High

Contributor guide

Open the contributing guide

Research direction

Start with code-analyzer-core PR #503 and the PMD process working-directory change, then reproduce with `sf code-analyzer rules --rule-selector pmd:MyRule` using `scanner/full.xml` and `scanner/subset.xml`. Done means relative rule references resolve from the workspace or configuration location, or a failed ruleset makes the run fail instead of exiting 0 with the engine silently removed.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, typescript
Domain
devtools, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.