apache / apache/lucene

"ant clean-jars" doesn't remove symlinks that point to non existent files [LUCENE-7830]

Open
#8,881 2 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

Discovered this helping ishan troubleshoot weird failures he was getting on his machine today.

Steps to reproduce...

```Java
$ touch /tmp/foo.jar
$ touch /tmp/bar.jar
$ ln -s /tmp/foo.jar solr/core/lib/
$ ln -s /tmp/bar.jar solr/core/lib/
$ ls solr/core/lib/

$ ls -l solr/core/lib/

18:10 bar.jar -> /tmp/bar.jar
18:10 foo.jar -> /tmp/foo.jar
$ rm /tmp/bar.jar
$ ant clean-jars
Buildfile: /home/hossman/lucene/dev/build.xml

clean-jars:

BUILD SUCCESSFUL
Total time: 0 seconds
$ ls -l solr/core/lib/
total 0
lrwxrwxrwx 1 hossman hossman 12 May 15 18:10 bar.jar -> /tmp/bar.jar
```

The situation that (i speculate) led to ishan's problems was (smething like) ...

1. at some point does a build that resolves all jars, ivy creates symlinks to it's cache
1. at some later point, either the ivy cache (or a subset of jars in it) gets blown away (possibly intentionally to save disk space) or the working dir is copied from one machine to another
1. the version of the dependency gets updated and/or the branch is changed so a diff version of some jar is needed - so "ant resolve" & all compilation and testing are happy even though there is a symlink to a file that isn't in the ivy cache anymore.
1. check-licenses will fail however, because when it attempts to resolve the list of all jar resources found by ant, the underlying file doesn't actaully exist, so the build fails with an error about an "old" jar that you don't expect to be used anyway...
```Java
// from LicenseCheckTask.java
if (!r.isExists()) {
throw new BuildException("JAR resource does not exist: " + r.getName());
}
```
1. `ant clean-jars` does nothing to fix the underlying problem

---
Migrated from [LUCENE-7830](https://issues.apache.org/jira/browse/LUCENE-7830) by Chris M. Hostetter (@hossman)

Contributor guide

Open the contributing guide

Research direction

Start with the clean-jars target in dev/build.xml and reproduce the dangling-symlink case from the issue. Read LicenseCheckTask.java and compare the behavior of ant clean-jars, ant resolve, and check-licenses; done means clean-jars removes the broken symlink so check-licenses no longer reports it.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.