packagecontrol / packagecontrol/thecrawl
Track file extensions for syntax definitions
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 19
- Forks
- 7
- Avg merge
- 4h 53m
- Merged PRs (30d)
- 14
Description
The rewrite opens up the possibility of a feature I'd been kicking around in my head for a couple years, though it might be too ambitious: I think it would be interesting and useful to index the file extensions that a package claims to provide syntax for.
Benefits
- PC can suggest packages from inside ST for un-highlighted (or wrongly, badly, etc. highlighted) files.
- It's another view into the package database, potentially exposing some additional utility for packages.
- The first time we do this, we may discover a bunch of packages that should have the
language syntaxtag, but don't.
Drawbacks
- Requires downloading, parsing, and saving results for packages. This can use a lot of API rate-limit.
- Mitigation: Maybe running only on
language syntaxpackages with new updates will drop the API / CPU requirements sufficiently. A one-time run of every package to fill in missing tags is probably still needed.
- Mitigation: Maybe running only on
Prior Art
- This feature does not exist in the current ST PC.
- VS Code's marketplace does track file extensions that could be highlighted. They don't need to do all the parsing, though, because the manifest file for a package tracks the extensions that it has syntax for. (Then Microsoft somewhat aggressively pushes those packages on people, which we would not want to do.)
Discussion
Taken from Discord
@kaste: I never had problems just searching for the actual "language" and then each of them should have the
language syntaxlabel. I often rather know that name (e.g. Clojure) before its file extension.
@michaelblyons:
- I don't actually trust the
language syntaxtag. Some packages have syntax definitions but not the tag.
- For some, we could one-time scan and add the tag to the appropriate Channel repo.
- But for others, arguably the language is kind of incidental to something else that the package is doing. I don't have strong opinions that we should add a tag if that is the case.
- There's also the case where it's not immediately apparent what the file extension is for. Granted, you could
$favorite_search_engineit and probably comes up with one of those ad-stuffedfile-extensions.infotype sites that gives you a good guess.- And lastly, there's the case where the name of the package might not signal well enough how much it does.
- PackageDev: If you find a
*.tmLanguagefile, you don't know to search for a Sublime configuration editing package because the internet tells you it's a TextMate file.- SSH Config has minimal syntaxes thrown in for known hosts, authorized keys, and a couple other things that don't suggest "Go look for SSH packages" to a user.
Disclaimer
I'm not going to be upset if the amount of work outweighs the benefit.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue proposes indexing file extensions by downloading and parsing package syntax definitions, but it does not name implementation files, tests, or entry points. First map the crawler's package scanning and result-storage flow, then define the package sources, parsing scope, rate-limit strategy, and a way to verify stored extension data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100