janestreet / janestreet/ppx_inline_test

Embed errors in the AST instead of raising

Open
#41 2 comments 0 reactions 0 assignees View on GitHub
forwarded-to-js-devs
Dominant language
OCaml
Stars
143
Forks
25
PR merge metrics
No merged PRs in 30d

Description

Currently, when `ppx_inline_test` encounters an error, it uses the `raise_errorf` function to raise a located error.

The exception is caught by `ppxlib`, which in this case:

- Catch the error,
- stops the rewriting process
- add the error (as a `[%%%ocaml.error ...]` extension node) to the _last valid ast_
- Use the resulting AST

The interruption of the rewriting is quite bad for the user experience! The implication for the users are:

- Since `ppx_inline_test` runs at the "context-free" phase, the "last valid AST" is before the context-free phase. So, no other derivers/extenders get run, which generates a lot of noise in the errors (such as "uninterpreted extensions" or "unbound identifiers")
- Only one (meaningful) error from your PPX is reported at a time.
Example

For instance:
```ocaml
let%test ("invalid1" [@tags "invalid"]) = false

let%test ("invalid2" [@tags "invalid"]) = false

let%test ("valid" [@tags "no-js"]) = false
```
would report several errors:
- `invalid is not a valid tag for inline tests` for `invalid1`: the right error
- `uninterpreted extension test` for `invalid2`: the wrong error, it should be as for `invalid1`
- `uninterpreted extension test` for `valid`: an error when there should not be one

You can find more information about error reporting in PPXs in [this section](https://ocaml.org/p/ppxlib/latest/doc/good-practices.html#handling_errors) of the ppxlib manual.

:question: Would you be willing to accept contributions to this issue? I'm considering assigning its resolution as part of an [outreachy](https://www.outreachy.org/) internship: see more information [here](https://github.com/ocaml-ppx/ppxlib/issues/389).

Contributor guide

Open the contributing guide

Research direction

Start with ppx_inline_test's current raise_errorf path and the ppxlib manual section on handling errors, focusing on the context-free rewriting phase and the AST error-extension behavior. Done means rewriting continues after each error, multiple meaningful errors are reported, and valid or later extensions do not produce spurious uninterpreted-extension errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
ocaml
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.