llvm / llvm/llvm-project

[clang-tidy] `verify-config` option gives false negatives

Open
#162,506 5 comments 0 reactions 0 assignees View on GitHub
clang-tidy false-negative
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

**Description**: Some invalid check options from `readability-identifier-naming` check are not triggering warnings from `--verify-config` command-line option with `clang-tidy`. It seems that every *type* from `readability-identifier-naming` check that does **not** have a `HungarianPrefix` option gives this error.

There may have other options from other checks causing this error, I can't really test them all.

From what I've seen, here's the list of the invalid check options that are not being detected as such when invoking the `--verify-config` clang-tidy command-line option.

- NamespaceHungarianPrefix
- InlineNamespaceHungarianPrefix
- ParameterPackHungarianPrefix
- GlobalFunctionHungarianPrefix
- ConstexprFunctionHungarianPrefix
- FunctionHungarianPrefix
- ConstexprMethodHungarianPrefix
- VirtualMethodHungarianPrefix
- ClassMethodHungarianPrefix
- PrivateMethodHungarianPrefix
- ProtectedMethodHungarianPrefix
- PublicMethodHungarianPrefix
- MethodHungarianPrefix
- TypedefHungarianPrefix
- TypeTemplateParameterHungarianPrefix
- ValueTemplateParameterHungarianPrefix
- TemplateTemplateParameterHungarianPrefix
- TemplateParameterHungarianPrefix
- TypeAliasHungarianPrefix
- MacroDefinitionHungarianPrefix
- ObjcIvarHungarianPrefix
- ConceptHungarianPrefix

**Branch**: `main`

**Steps to reproduce**:
```bash
clang-tidy --verify-config --config="{ Checks: 'readability-identifier-naming', CheckOptions: { readability-identifier-naming.ConceptHungarianPrefix: 'Off' } }" # That will not trigger a warning as it should
clang-tidy --config="{ Checks: 'readability-identifier-naming', CheckOptions: { readability-identifier-naming.ConceptHungarianPrefix: 'Off' } }" test.cpp # This will trigger a warning, telling us that readability-identifier-naming.ConceptHungarianPrefix is not a valid option
```
With `test.cpp` being a placeholder C++ file, it doesn't matter what it contains.

Of course, `ConceptHungarianPrefix` can be replaced by any option listed above.

I've made a small Python script that prints the invalid options that are not detected as such (that's how I got the list):

```python
import subprocess

clang_tidy_path =
options = ["Namespace", "InlineNamespace", "EnumConstant", "ScopedEnumConstant", "ConstexprVariable", "ConstantMember", "PrivateMember", "ProtectedMember", "PublicMember", "Member", "ClassConstant", "ClassMember", "GlobalConstant", "GlobalConstantPointer", "GlobalPointer", "GlobalVariable", "LocalConstant", "LocalConstantPointer", "LocalPointer", "LocalVariable", "StaticConstant", "StaticVariable", "Constant", "Variable", "ConstantParameter", "ParameterPack", "Parameter", "PointerParameter", "ConstantPointerParameter", "AbstractClass", "Struct", "Class", "Union", "Enum", "GlobalFunction", "ConstexprFunction", "Function", "ConstexprMethod", "VirtualMethod", "ClassMethod", "PrivateMethod", "ProtectedMethod", "PublicMethod", "Method", "Typedef", "TypeTemplateParameter", "ValueTemplateParameter", "TemplateTemplateParameter", "TemplateParameter", "TypeAlias", "MacroDefinition", "ObjcIvar", "Concept"]

for option in options:
config = "{ Checks: 'readability-identifier-naming', CheckOptions: { readability-identifier-naming." + option + "HungarianPrefix: 'Off' } }"
verify_result = subprocess.run([clang_tidy_path, "--verify-config", f"--config={config}"], capture_output=True)
run_result = subprocess.run([clang_tidy_path, f"--config={config}", "./test.cpp"], capture_output=True)
verify_ok = verify_result.stdout == b'No config errors detected.\n'
run_ok = run_result.stdout != b"warning: invalid identifier naming option '" + option.encode("utf-8") + b"HungarianPrefix' [clang-tidy-config]\n"
if verify_ok != run_ok:
print(option)
```

**Expected result**: The two commands should be coherent and give a warning.

**Actual result**: `--verify-config` does not warn about some invalid check options.

**Additional notes**: I'm not yet familiar with `clang-tidy` source code but if you think this is fine for me to try to fix this issue, I would be glad to try to. I would like to become familiar with clang-tidy source code and I feel like this would be a good start to. What do you think?

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the mismatch with the two clang-tidy commands in the issue, using the listed invalid readability-identifier-naming options. Trace how --verify-config handles check options compared with normal clang-tidy validation. Done means every invalid option produces a coherent warning in both paths, including the listed options.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.