Macaulay2 / Macaulay2/M2

A doc node's recorded location points at whatever follows it; make doc a keyword as TEST is

Open
#4,599 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bugs directory Documentation
Dominant language
Macaulay2
Stars
435
Forks
297
Avg merge
4d 20h
Merged PRs (30d)
11

Description

This issue was triaged from [`bugs/dan/1-positions-of-strings`](https://github.com/Macaulay2/M2/blob/388c1ff0ce30d83751dea7bc7eac77fdc1305dd7/bugs/dan/1-positions-of-strings), one of the 857 files removed from the pre-GitHub `bugs/` tree by [`d2c8d27826`](https://github.com/Macaulay2/M2/commit/d2c8d27826) and catalogued in [#36](https://github.com/Macaulay2/M2/issues/36). **The commentary below was written by Claude (Claude Opus 5, via Claude Code)**, not by @d-torrance, whose account posted it -- please weigh it accordingly.

### The original file, verbatim

```text
It would be nice if strings formed with quotations marks knew their position
(filename, line number, column number). Same for strings returned by "get". And
"lines" and "substring" should preserve the positions. That would make the code
in SimpleDoc much simpler, and it would mean that error messages from "value"
would generally be much more informative.

This could be implemented by a new Type of String called LocatedString.
```

### Where it stands today

The note above proposes a `LocatedString` type. This issue files the need it names rather than that
remedy, because M2 has since grown a cheaper mechanism for exactly this and one of the note's two
motivations can be demonstrated as a plain defect.

### A documentation node's recorded location points at the next construct

Same file, two adjacent blocks:

```m2
1 newPackage("ZzLoc2", DebuggingMode => true)
2 export {"zzThree"}
3 zzThree = () -> 3
4 beginDocumentation()
5 doc ///
6 Key
7 zzThree
8 Headline
9 the third one
10 ///
11 TEST ///
12 assert(zzThree() == 3)
13 ///
```

```
locate makeDocumentTag zzThree --> ZzLoc2.m2:11:0 <- line 11 is the TEST line
(ZzLoc2#"test inputs")#0#"location" --> ZzLoc2.m2:11:5-13:3 <- exact span
```

The `doc` node occupies lines 5–10 and reports line 11 — not off by one, but pointing at whatever
happens to follow the block. The `TEST` immediately below it reports its true start and end.
A `document { ... }` node fares better but is still wrong: one on line 8 reports line 9.

### Why `TEST` gets it right

Because it is a keyword. `binding.d:315`:

```
export TestS := special("TEST", unaryop, precSpace, wide);
```

and `testing.m2:23-24` states the mechanism:

> `-- TEST is a keyword that takes an object as input and determines its location.`
> `-- It then passes the object and its location to addTest.`

`addTest(String, FilePosition)` then stores `"location" => loc`. That arrived in
[#3419](https://github.com/Macaulay2/M2/pull/3419) (2024-08-31).

The general fact underneath: `Code` in the interpreter knows its own location, while the `Expr` that
reaches a top-level function like `doc` does not. A keyword sees the former.

### The suggestion

Make `doc` a keyword, so a documentation node gets a `FilePosition` spanning its block the way a test
does. [#3599](https://github.com/Macaulay2/M2/pull/3599) ("User-defined keywords") is the
infrastructure that would make this cheap rather than bespoke — it moves keyword creation to the M2
level and gives keywords a default operation that looks up the corresponding method.

### The bookkeeping this would replace

`SimpleDoc.m2` carries 55 mentions of `linenum`/`lineNumber`, tracking line numbers by hand:
`makeTextline(line, linenum)` pairs each line with its number, and `keylinenum` is threaded through
roughly twenty section handlers. The 2009 note predicted that a located string would make this
unnecessary; a located `doc` block would do the same for the node's own position.

### What this reframing does not cover

The note's other motivation is separate and remains: `value` on a string reports a position *within*
the string against a synthetic filename, with no provenance.

```
i1 : value "1 + \n * 2"
currentString:2:2:(3): error: no method for prefix operator * applied to object:
```

Line and column, but nothing saying where the string came from. That part would still want something
like the original proposal, or an optional position argument to `value`.

### Related

[#3420](https://github.com/Macaulay2/M2/issues/3420) wants a `Test` keyword *inside* `document`/`doc`
for coverage tracking — same family, different purpose.
[#4409](https://github.com/Macaulay2/M2/issues/4409) is about templates for documentation nodes.
Neither proposes making `doc` itself a keyword.

`open` · disposition `issue` · source of truth: [`bug-triage/catalog.tsv`](https://github.com/d-torrance/M2/blob/bug-triage/bug-triage/catalog.tsv)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with binding.d:315, testing.m2:23-24, and the user-defined keyword infrastructure from #3599. Read SimpleDoc.m2 to understand the existing linenum and lineNumber bookkeeping, then compare doc and TEST locations using the ZzLoc2 example. Done means a doc node records a FilePosition spanning its own block rather than the following construct, with existing documentation behavior preserved.

Written by the indexing model from the issue text.

Assessment

Domain
compilers, documentation
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.