leanprover / leanprover/lean4

RFC: Teach lake to use github error formats

Open
#8,648 1 comment 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Lake P-medium RFC
Dominant language
Lean
Stars
9.2k
Forks
990
Avg merge
1d 17h
Merged PRs (30d)
175

Description

Proposal

Github has special support for setting error messages with

::error file={name},line={line},col={col},endLine={endLine},endColumn={endColumn},title={title}::{message}

where messages should be url-escaped (in particular, replacing newlines with %0A).

If lake learnt a --github flag, then github would automatically show multi line error messages on the lines that they originate from, such as

We had code for this in mathlib 3 via lean --json (added in https://github.com/leanprover-community/mathlib3/pull/5726), but this is not exposed via lake (as far as I can tell), so writing such a wrapper is not possible.

Currently the workaround is to use a "problem matcher" for github actions (https://github.com/leanprover-community/mathlib4/pull/24316), but these do not support the multi-line error messages produced by Lean, and so we see 'could not synthesize' but then the rest of the message is cut off.

It would be possible to have lake detect if it is running on github instead of having an explicit --github flag, but I expect this would just be confusing to users.

Maintainability

Lake already maintains a record of (severity, message) pairs, which can be converted to either ANSI or plain strings. Implementing this PR would involve:

  • Adding a third state beyond ANSI and plain, "github".
  • Either storing the full position message in the lake log message (via Options, as some messages can come from lake itself), or parsing it back out of the message strings
Community Feedback

#mathlib4 > gh-problem-matcher-wrap @ 💬

Impact

Add 👍 to issues you consider important. If others benefit from the changes in this proposal being added, please ask them to add 👍 to it.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by inspecting lake's existing ANSI and plain log formatting, then read the linked mathlib3 scripts/detect_errors.py and the referenced GitHub Actions error format. The change would be complete when lake can emit correctly escaped multiline GitHub annotations with source positions while preserving existing output modes.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions
Domain
build-system, ci-cd, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.