babel / babel/babel

Support proper JSDoc block AST

Open
#14,449 8 comments 0 reactions 0 assignees View on GitHub
i: needs triage
Dominant language
TypeScript
Stars
44k
Forks
6k
Avg merge
5d 15h
Merged PRs (30d)
23

Description

### 💻

- [ ] Would you like to work on this feature?

### What problem are you trying to solve?

Besides being a virtual standard for documentation, JSDoc has been blessed further by TypeScript in its using it to encode type information (in JavaScript mode).

(JSDoc could even be used in theory to be used as a basis for conversion to TypeScript which could in turn be converted into WebAssembly through AssemblyScript, providing a role beyond documentation and type checking into actual running code.)

However, until now there has been no standard way to represent JSDoc blocks, beyond treating them just as other comments, in AST.

In `eslint-plugin-jsdoc`, in conjunction with via [jsdoccomment](https://github.com/es-joy/jsdoccomment) , we use comment AST to allow rules to be defined which target particular contexts based on their AST structure. We do this so that users have full control of contexts to which rules apply, and so that they can require or prevent certain JSDoc structures from existing. A user could therefore require certain tags but only where a particular tag was present, or report that a particular type was used or shouldn't be used as the child of another type. These types include, via [`jsdoc-type-pratt-parser`](https://github.com/jsdoc-type-pratt-parser/jsdoc-type-pratt-parser), support for any TypeScript types that are used within JSDoc. In other words, any aspect of a JSDoc block can be targeted and targeted intelligently.

In a proof of concept ESLint parser, at https://github.com/es-joy/jsdoc-eslint-parser , I use `@babel/eslint-parser` (or optionally `@typescript-eslint/parser`) to get `jsdoc` and `jsdocBlocks` properties added which point to JSDoc AST (these are analogous to `leadingComments`/`trailingComments` and `comments` respectively, but for JSDoc). Using a full-blown parser makes possible AST queries which, unlike our `eslint-plugin-jsdoc` hack, can combine JSDoc as well as regular JavaScript AST into a single expression, and be targeted by ESLint rules such as `no-restricted-syntax`. This AST could in theory be targeted by parser-specific ESLint rules as well.

What this doesn't get us is the ability to manipulate AST, given that there apparently are not ESTree-capable tools which support such AST-aware manipulation as with `@babel/traverse`.

A specific use case we have with ESLint (and it is an approach I'd like to use in my own projects as well) is that we'd like to have certain directives in our JSDoc which can be stripped or modified before being supplied to TypeScript, so that we can keep our source code in plain JavaScript + JSDoc, while getting a declaration file built and one that doesn't need editing separately from the JavaScript, with its type information derived solely from inline JSDoc. (TypeScript's awareness of JSDoc + plain JavaScript, though good, doesn't currently provide quite as much robustness as regular TypeScript when in plain JSDoc+JavaScript mode, so we want to work around some of those limitations in our build process, for example, by stripping or expanding typedefs which TypeScript always exports, though we just want to use them as internal aliases.)

I imagine there may be other use cases for transformation. Some might wish to convert JSDoc-enhanced JavaScript into asm.js, or as mentioned, into proper TypeScript for use by AssemblyScript, causing a pipeline from regular JavaScript to WebAssembly code. With ESLint's awareness of such comments, rules could be made to require or prevent certain syntax or structures for typing purposes (e.g., reporting against using the "number" type instead of a particular float/integer type).

### Describe the solution you'd like

So besides my [discussion question](https://github.com/babel/babel/discussions/14445) asking whether you might align more with ESTree (so your types and generator could work out of the box), I'm especially wondering if you would be open to supporting optional parsing and support of special JSDoc block AST properties which could be understood in code traversal, manipulation and generation (if not builders as well). We have an algorithm, originally used in ESLint, for determining the attachment points of a JSDoc node if that may help.

While many will no doubt continue to favor the more succinct form that TypeScript proper provides, it is undoubtedly compelling for projects to maintain accessibility to a greater developer base by allowing plain JavaScript which can be progressively enhanced in a standard, simple way, whereby novices can more easily set aside concern of the JSDoc, while learning to make use of it or allowing others on the type to do so.

If such a pipeline existed, existing code might already be ready for type awareness as well.

### Describe alternatives you've considered

One alternative is to fork Babel types and Babel generator to allow us to manipulate the JSDoc AST we have with the experimental `@es-joy/jsdoc-eslint-parser` and then reserialize it. This wouldn't as easily allow others to benefit from the work, or ourselves to benefit as easily from future improvements. It also requires some extra work to get familiar with Babel, to make a fork, deal with the TypeScript source which is less familiar, etc.

Another alternative is to build our own ESTree-aware manipulating tools for use with our parser, but this would require even more work.

Yet another alternative is just using Babel as it is, and applying regular expressions against the stringified comments. Such manipulation can be unpleasant and prohibitive.

Yet another alternative is using Babel as it is, but manipulating the comments and then reserializing them back as regular comments (assuming Babel types' manipulation supports modification and then reserialization of comments too). This is probably the route I would take if there is no interest by your project, but besides this being less elegant than solely manipulating and serializing the JSDoc AST , it seems that it may be compelling for other use cases for JSDoc blocks to be treated as (optionally) first class citizens by the parser, manipulators and generator, especially given TypeScript and AssemblyScript applications.

### Documentation, Adoption, Migration Strategy

I can only think of adding an option like `structuredJsdoc: true` to the parser which triggers attachment of such structured comment nodes.

`@babel/types` might also expose builders like `t.jsdocBlock()`, `t.jsdocTag()`, `t.jsdocType()`, `t.jsdocDescriptionLine`, etc., but your traversal, manipulation methods, generator, etc. should require no public changes requiring any special documentation.

I realize this may be rather ambitious, but I thought I'd spell about some of the potential appeal in case you may be willing to adopt. Thanks!

Contributor guide

Open the contributing guide

Research direction

Start by reviewing Babel's parser, @babel/types, traversal, and generator support alongside the linked es-joy/jsdoc-eslint-parser proof of concept. Compare the proposed structuredJsdoc option and JSDoc node types with existing comment AST handling. Done would require an agreed design for parsing, attaching, traversing, manipulating, and generating JSDoc blocks.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, typescript
Domain
compilers
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.