stan-dev / stan-dev/stanc3

Simplify parser

Open
#614 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

cleanup parsing
Dominant language
OCaml
Stars
160
Forks
59
Avg merge
21h 45m
Merged PRs (30d)
26

Description

Simplify parser

The parser distinguishes between var_decl, top_var_decl, and top_var_decl_no_assign even though the AST does not. There's no logical need for the grammar to have three separate rules; semantic check could handle the potential syntax errors.
For instance, Stan does not allow tilde-statements outside of the model block but there's no special statement_no_tilde rule in the grammar for other blocks. So this fails to parse

parameters {
  real x;
  x ~ std_normal(); // only decls allowed
}

but this parses successfully and fails in semantic check

parameters {
  real x;
}
generated quantities {
  x ~ std_normal(); // any statement parses
}

Parser errors are defined in parser.messages. That file is not easy to work with. Currently it is incomplete because I didn't know I was supposed add some error states there in PR #573
The parser error messages have no context other than the parser state that failed. We decided to fix #469 by changing the error message. The message relevant is here:
https://github.com/stan-dev/stanc3/blob/458a7933728debdb8b3ce14e5af6960faef95513/src/frontend/parser.messages#L1392-L1402
The parser state 661 corresponds to any unexpected symbol in data or parameters block. As far as I know the only way to give a specific error for a semicolon is by creating a new parser state with the UNREACHABLE hack. On the other hand, if this were handled in Semantic_check the change would be trivial.

So I'm proposing changing the parser to accept any program that AST can represent and moving all other checks to Semantic_check.

@rybern what do you think?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing the grammar rules for var_decl, top_var_decl, and top_var_decl_no_assign with parser.messages and the Semantic_check entry points. Review the examples in this issue and the linked discussions for #469, #573, and #581; done means AST-representable programs parse while remaining restrictions are reported by semantic checking, with relevant parser messages updated.

Written by the indexing model from the issue text.

Assessment

Tech stack
ocaml
Domain
compilers
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.