apache / apache/lucene

Replace (or accelerate) check-broken-links.gradle with a doclet pass [LUCENE-9496]

Open
#10,535 4 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Minor type:task
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

This is just a placeholder, but perhaps somebody will find the time to push this forward. The current python script in check-broken-links reparses all emitted HTML files to find links. I have a strong feeling this could be done better.

Javadoc doclets have access to parse trees for both the code and the javadoc comments (including information about HTML tags, code links, etc.). For example, this information is used by the built-in javac HTML linter.

Maybe we could replace the python linter entirely; verify where code links will point at, where existing HTML links point to and validate this information. I wrote some of that link-parsing code in Carrot2 (to convert javadocs into a structured JSON format used in other documentation). The code there is free to eyeball and borrow, if needed.

https://docs.oracle.com/en/java/javase/11/docs/api/jdk.compiler/com/sun/source/util/DocTreeScanner.html
https://github.com/carrot2/carrot2/blob/master/infra/jsondoclet/src/main/java/com/carrotsearch/jsondoclet/JavaDocsVisitor.java#L135
https://github.com/carrot2/carrot2/blob/master/infra/jsondoclet/src/main/java/com/carrotsearch/jsondoclet/PlainReferenceConverter.java

---
Migrated from [LUCENE-9496](https://issues.apache.org/jira/browse/LUCENE-9496) by Dawid Weiss (@dweiss), updated Sep 02 2020
Linked issues:
- #10255

Contributor guide

Open the contributing guide

Research direction

Start by examining check-broken-links.gradle and the current Python script in check-broken-links, then read the DocTreeScanner reference and the linked Carrot2 JavaDocsVisitor and PlainReferenceConverter examples. Determine whether a doclet pass can replace the HTML reparsing approach and validate code and HTML link targets; done means the replacement or acceleration is implemented and its link checks are verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, python
Domain
build-system, documentation, tooling
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.