forcedotcom / forcedotcom/code-analyzer
[BUG][code-analyzer] 5.16.0 silently drops custom PMD rulesets that point at a sibling file by relative path
- Dominant language
- TypeScript
- Stars
- 240
- Forks
- 52
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 5
Description
### Have you tried to resolve this issue yourself first?
- [x] I confirm I have gone through the above steps and still have an issue to report.
### Bug Description
After upgrading to `@salesforce/plugin-code-analyzer` 5.16.0, a scan that used to run our custom PMD rules stops running them, and nothing fails. The exit code is still 0. The only sign is one Critical violation attached to no file, with a summary of `Found 1 violation(s) across 0 file(s)`. A CI gate that does not use `--severity-threshold` reads green while zero Apex rules ran.
`sf code-analyzer config` and `sf code-analyzer rules` show the cause: `Cannot resolve rule/ruleset reference 'scanner/full.xml/MyRule'`.
**Why it happens.** Our repo has two custom rulesets: a full one that defines the rules, and a smaller one that reuses some of them with ``, a path relative to the repo root. Up to 5.15.x, PMD resolved that path against the repo being scanned, because the PMD process was started from there.
code-analyzer-core PR #503 (the CWE-427 hardening) now starts the PMD process from the engine's own install folder (`cwd: __dirname`). The PR calls this behaviour-preserving because every argument handed to `java` is an absolute path. That is true for the paths the engine passes, but not for paths written inside a custom ruleset: PMD still resolves those from the process's working directory, which is now the wrong folder. The whole ruleset then fails to load, and the PMD engine drops out of the run.
The 5.16.0 release notes do not mention it.
Originally filed on the core repo as forcedotcom/code-analyzer-core#508. Cross-filing here because that repo's issue bot reports its config as incomplete, so the report never reached triage.
### Output / Logs
```
› Error: Code Analyzer [8:43:51.703]:
› Failed to get rules from engine with name 'pmd' due to the following
› error:
› PMD errored when attempting to load a custom ruleset
› "scanner/full.xml/MyRule". Make sure the
› resource is a valid ruleset file on disk or on the Java classpath.
›
› PMD Exception:
› | Cannot load ruleset
› scanner/full.xml/MyRule: Cannot resolve
› rule/ruleset reference
› 'scanner/full.xml/MyRule'. Make sure the
› resource is a valid file or URL and is on the CLASSPATH. Use --debug (or a
› fine log level) to see the current classpath.
›
› If you wish to ignore this error and disable this engine, then update
› your Code Analyzer configuration with:
› engines:
› pmd:
› disable_engine: true
=== Summary
Found 1 violation(s) across 0 file(s):
1 Critical severity violation(s) found.
```
The command exits 0.
### Steps To Reproduce
1. `scanner/full.xml` defines `MyRule`. `scanner/subset.xml` contains ``.
2. `code-analyzer.yml` in the repo root lists both files under `engines.pmd.custom_rulesets`.
3. Run `sf code-analyzer rules --rule-selector pmd:MyRule` from the repo root.
On 5.15.x the rule is listed. On 5.16.0 the ruleset fails to load with the error above. The same configuration loaded cleanly on 5.14.0 and 5.15.0.
### Expected Behavior
Either of these:
- Paths inside a custom ruleset resolve against the config file's folder or the workspace root, as they did before 5.16.0. One way: keep the working directory pinned, but add the workspace root to PMD's classpath, so `scanner/...` resolves as a classpath resource.
- Or the `custom_rulesets` docs and the 5.16.0 release notes state that a custom ruleset must be self-contained (no relative `` to another file).
In both cases, a ruleset that fails to load should fail the run, not exit 0 with the engine silently removed.
### Operating System
macOS 26.6.2 (build 25G83), Apple Silicon
### Salesforce CLI Version
@salesforce/cli/2.150.6 darwin-arm64 node-v24.19.0
### Code Analyzer Plugin (code-analyzer) Version
code-analyzer 5.16.0 (code-analyzer-core 0.53.0, code-analyzer-pmd-engine 0.46.0, PMD 7.26.0)
### Node Version
v22.22.3
### Java Version
openjdk version "26.0.2.1" 2026-08-18
### Python Version
_No response_
### Additional Context (Screenshots, Files, etc)
The bundled PMD is 7.26.0 on both 5.15.0 and 5.16.0, so this is not a PMD change. The only relevant difference between the two plugin releases is the working directory of the spawned `java` process.
### Workaround
Generate the smaller ruleset as a self-contained copy of the rules it needs, instead of pointing at the full one. That is what we now do, guarded by a test that rejects any file-path `` in our rulesets.
### Urgency
High
Contributor guide
Research direction
Start with code-analyzer-core PR #503 and the PMD process working-directory change, then reproduce with `sf code-analyzer rules --rule-selector pmd:MyRule` using `scanner/full.xml` and `scanner/subset.xml`. Done means relative rule references resolve from the workspace or configuration location, or a failed ruleset makes the run fail instead of exiting 0 with the engine silently removed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, typescript
- Domain
- devtools, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100