github-linguist / github-linguist/linguist

Suggestion: Move all grammar-related metadata to `grammars.yml`

Open
#4,990 10 comments 1 reaction 1 assignee Claimed by @Alhadis View on GitHub
Dominant language
Ruby
Stars
13.7k
Forks
5.4k
Avg merge
1d 15h
Merged PRs (30d)
17

Description

Currently, metadata pertaining to grammars and case-by-case exceptions are handled in four different places:

* [`.gitmodules`](https://github.com/github/linguist/blob/b7f579597050aaf7ac19c6c6ec504253a577ff13/.gitmodules) specifies URLs.
* [`grammars.yml`](https://github.com/github/linguist/blob/b7f579597050aaf7ac19c6c6ec504253a577ff13/grammars.yml) specifies scopes.
* [`tools/grammars/compiler/data.go`](https://github.com/github/linguist/blob/16582708fe8a35883862143fbdf5eb1ec843d6f1/tools/grammars/compiler/data.go) specifies scope-maps and case-by-case overrides
* [`vendor/licenses/config.yml`](https://github.com/github/linguist/blob/96ad1185828f44bb9b774328a584551ee57ed264/vendor/licenses/config.yml#L16-L22) identifies (for Licensee) which grammars had their licenses manually-reviewed

Moreover, the difficulties [related](https://github.com/github/linguist/issues/4974#issuecomment-684577601) by @pastra98 made me realise there's more we could be doing with regards to locating grammar and license files. Specifically, we should be able to provide a manual path if need be — the [currently hardcoded](https://github.com/github/linguist/blob/ec800d3887d8d635b7a1d9053076bee35431b757/tools/grammars/compiler/loader.go#L106-L129) search locations can remain the default for grammars without a `src:` field defined, or whatever.

Here's how it *might* look.

~~~yml
# Each entry correlates to a directory in "vendor/grammars/#{key}"
--
abl-tmlanguage:
license: MIT
source: chriscamicas/abl-tmlanguage
scopes:
- source-abl

# Location of files inside submodule repository. Usually
# calculated automatically, though maybe it makes sense
# to always provide this list, as opposed to only those
# grammars with non-standard file locations?
files:
grammar: abl.tmLanguage.json
license: LICENSE

actionscript3-tmbundle:
license: MIT
source: simongregory/actionscript3-tmbundle
scopes:
- source.actionscript.3
- text.html.asdoc
- text.xml.flex-config

c.tmbundle:
license: MIT
source: textmate/c.tmbundle
scopes:
- source.c
- source.c++
- source.c.platform
aliases:
source.c++: source.cpp

hy.tmLanguage:
license: MIT
source: Slowki/hy.tmLanguage
scopes:
- source.hy
paths:
- hy.json
- LICENSE.md

language-roff:
license: ISC
source: Alhadis/language-roff
scopes:
- hidden.manref
- source.ditroff
- source.ditroff.desc
- source.gremlin
- source.ideal
- source.pic
- text.roff
- text.runoff

sublimesystemverilog:
license: MIT
source: https://bitbucket.org/Clams/sublimesystemverilog/get/default.tar.gz
scopes:
- source.systemverilog
- source.ucfconstraints

Genshi.tmbundle:
license: MIT
source: https://svn.edgewall.org/repos/genshi/contrib/textmate/Genshi.tmbundle/Syntaxes/Markup%20Template%20%28XML%29.tmLanguage
scopes:
- text.xml.genshi
~~~

Thoughts?

**EDIT:** Oh yeah, it'd also be nice if we refined our terminology a little, because it's confusing to refer to both a TextMate compatible grammar file **and** the submodule containing it as a _"grammar"_ (maybe _"grammar-source"_ for the latter?) Given most of the grammars I write nowadays are almost exclusively added to `language-etc` (a super-bundle of whatever I can't be fucked publishing separately anymore, but nothing specific), it'd be a helpful distinction.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.