NullnessChecker: java.lang.NullPointerException: Cannot invoke "com.sun.source.tree.Tree.getKind()" because "tree" is null

Dear human checkers,

the NullnessChecker, somewhat ironically, has a null dereference bug. ;)

I run into the following exception

error: Error when invoking constructor org.checkerframework.checker.nullness.KeyForAnnotatedTypeFactory(class org.checkerframework.common.basetype.BaseTypeChecker) on args [org.checkerframework.checker.nullness.KeyForSubchecker@6bf08014]; cause: Cannot invoke "com.sun.source.tree.Tree.getKind()" because "tree" is null
  ; The Checker Framework crashed.  Please report the crash.
  Exception: java.lang.NullPointerException: Cannot invoke "com.sun.source.tree.Tree.getKind()" because "tree" is null; java.lang.NullPointerException: Cannot invoke "com.sun.source.tree.Tree.getKind()" because "tree" is null
        at jdk.compiler/com.sun.source.util.TreePath.<init>(TreePath.java:118)
        at jdk.compiler/com.sun.source.util.TreePath.<init>(TreePath.java:109)
        at jdk.compiler/com.sun.source.util.TreePath.getPath(TreePath.java:50)
        at org.checkerframework.framework.type.AnnotatedTypeFactory.getPath(AnnotatedTypeFactory.java:3731)
        at org.checkerframework.framework.util.defaults.QualifierDefaults.nearestEnclosingExceptLocal(QualifierDefaults.java:431)
        at org.checkerframework.framework.util.defaults.QualifierDefaults.applyDefaults(QualifierDefaults.java:513)
        at org.checkerframework.framework.util.defaults.QualifierDefaults.annotate(QualifierDefaults.java:417)
        at org.checkerframework.framework.type.GenericAnnotatedTypeFactory.addComputedTypeAnnotations(GenericAnnotatedTypeFactory.java:1855)
        at org.checkerframework.framework.type.GenericAnnotatedTypeFactory.addComputedTypeAnnotations(GenericAnnotatedTypeFactory.java:1773)
        at org.checkerframework.framework.type.AnnotatedTypeFactory.getAnnotatedTypeFromTypeTree(AnnotatedTypeFactory.java:1395)
        at org.checkerframework.framework.type.SupertypeFinder$SupertypeFindingVisitor.supertypesFromTree(SupertypeFinder.java:329)
        at org.checkerframework.framework.type.SupertypeFinder$SupertypeFindingVisitor.visitDeclared(SupertypeFinder.java:186)
        at org.checkerframework.framework.type.SupertypeFinder.directSupertypes(SupertypeFinder.java:60)
        at org.checkerframework.framework.type.AnnotatedTypeMirror$AnnotatedDeclaredType.directSupertypes(AnnotatedTypeMirror.java:1072)
        at org.checkerframework.framework.stub.AnnotationFileParser.annotateSupertypes(AnnotationFileParser.java:1200)
        at org.checkerframework.framework.stub.AnnotationFileParser.processType(AnnotationFileParser.java:1128)
        at org.checkerframework.framework.stub.AnnotationFileParser.processTypeDecl(AnnotationFileParser.java:969)
        at org.checkerframework.framework.stub.AnnotationFileParser.processCompilationUnit(AnnotationFileParser.java:793)
        at org.checkerframework.framework.stub.AnnotationFileParser.processStubUnit(AnnotationFileParser.java:769)
        at org.checkerframework.framework.stub.AnnotationFileParser.process(AnnotationFileParser.java:758)
        at org.checkerframework.framework.stub.AnnotationFileParser.parseStubFile(AnnotationFileParser.java:658)
        at org.checkerframework.framework.stub.AnnotationFileElementTypes.parseAnnotationFiles(AnnotationFileElementTypes.java:286)
        at org.checkerframework.framework.stub.AnnotationFileElementTypes.parseStubFiles(AnnotationFileElementTypes.java:186)
        at org.checkerframework.framework.type.AnnotatedTypeFactory.parseAnnotationFiles(AnnotatedTypeFactory.java:3864)
        at org.checkerframework.framework.type.GenericAnnotatedTypeFactory.postInit(GenericAnnotatedTypeFactory.java:403)
        at org.checkerframework.checker.nullness.KeyForAnnotatedTypeFactory.<init>(KeyForAnnotatedTypeFactory.java:93)
        at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:67)
        at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:484)
        at org.checkerframework.common.basetype.BaseTypeChecker.invokeConstructorFor(BaseTypeChecker.java:338)
        at org.checkerframework.common.basetype.BaseTypeVisitor.createTypeFactory(BaseTypeVisitor.java:322)
        at org.checkerframework.common.basetype.BaseTypeVisitor.<init>(BaseTypeVisitor.java:271)
        at org.checkerframework.common.basetype.BaseTypeVisitor.<init>(BaseTypeVisitor.java:260)
        at org.checkerframework.common.basetype.BaseTypeChecker.createSourceVisitor(BaseTypeChecker.java:260)
        at org.checkerframework.common.basetype.BaseTypeChecker.createSourceVisitor(BaseTypeChecker.java:92)
        at org.checkerframework.framework.source.SourceChecker.initChecker(SourceChecker.java:907)
        at org.checkerframework.common.basetype.BaseTypeChecker.initChecker(BaseTypeChecker.java:114)
        at org.checkerframework.common.basetype.BaseTypeChecker.initChecker(BaseTypeChecker.java:107)
        at org.checkerframework.framework.source.SourceChecker.typeProcessingStart(SourceChecker.java:858)
        at org.checkerframework.javacutil.AbstractTypeProcessor$AttributionTaskListener.finished(AbstractTypeProcessor.java:165)
        at jdk.compiler/com.sun.tools.javac.api.ClientCodeWrapper$WrappedTaskListener.finished(ClientCodeWrapper.java:876)
        at jdk.compiler/com.sun.tools.javac.api.MultiTaskListener.finished(MultiTaskListener.java:132)
        at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1408)
        at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1365)
        at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:960)
        at jdk.compiler/com.sun.tools.javac.main.Main.compile(Main.java:317)
        at jdk.compiler/com.sun.tools.javac.main.Main.compile(Main.java:176)
        at jdk.compiler/com.sun.tools.javac.Main.compile(Main.java:64)
        at jdk.compiler/com.sun.tools.javac.Main.main(Main.java:50)

The issue occured when I was trying to "astub" some generated code in a large codebase, to modify some @NonNull return values in the gerneated code (which is being compiles along with my non-generated code) to @Nullable ones.

The problem boils down to the following minimal case. Consider sources src/Intf.java and src/Impl.java, and two stubs astubs/Intf.astub and astubs/Impl.astub.

$ cat src/*
public class Impl implements Intf {
    public String read() {
        return null;
    }
}

public interface Intf {
    String read();
}

$ cat astubs/*
import org.checkerframework.checker.nullness.qual.Nullable;

public class Impl implements Intf {
    public @Nullable String read();
}

import org.checkerframework.checker.nullness.qual.Nullable;

public interface Intf {
    @Nullable String read();
}

When I run the NullnessChecker over these (giving -Astubs=astubs) I trigger the crash.

The assertion at AnnotatedTypeFactory.getPath(AnnotatedTypeFactory.java:3711), that the current root cannot be null is incorrect, so I guess the annotation should rather be a guard against that case.

In line AnnotatedTypeFactory:187, the root is actually declared @Nullable, but the comment above seems to indicate that I have hit a corner case that isn't covered in the test suite yet...

  /** Optional! The AST of the source file being operated on. */
  // TODO: when should root be null? What are the use cases?
  // None of the existing test checkers has a null root.
  // Should not be modified between calls to "visit".
  protected @Nullable CompilationUnitTree root;

I'll attach a tarball with the above files and a build.sh script for easy reproduction of the problem. You may have to adjust paths in it, of course.

I didn't have enough time to have a look at the test suite yet, to make this into a test case, but I assume you'll probably beat me to it. As for the root cause: My bet is that astub files (or astub-directories?) just aren't considered as part of the source tree.

Thanks for your awesome work over the years!
Flexximilian

checkerframework-issue-5842.tar.gz