coala / coala/coala-quickstart

Discussion: Scope of Info Extraction Classes

Open
#117 1 comment 0 reactions 0 assignees View on GitHub
question status/STALE
Dominant language
Python
Stars
49
Forks
74
PR merge metrics
No merged PRs in 30d

Description

The Info Extractor classes ([cEP-0009](https://github.com/coala/cEPs/blob/master/cEP-0009.md)) are used to extract information from project files and use that information to our advantage.

Currently, in the GSoC project "[Enhance coala-quickstart](http://projects.coala.io/#/projects?project=enhance_coala-quickstart)", the utilization of the extracted information will be in the form of creating coafile sections with some setting values already discovered from this info and we are considering only the files that provide such information. Right now, the utilization is very specific to coala-quickstart and so me and my mentors decided to have these "Info Extractor" classes in quickstart only, but the scope for the usage of extracted information is much wider and hence there are some sound reasons for which these classes may be migrated to coalib in future.

Some of the ideas for usage of these classes include

- Extracting code to lint from project files. For example, the "scripts" field in `package.json` file contains bash commands and we may want to pass them through `ShellCheckBear`. So this way, we will be able to lint shellscript code inside a json file (couldn't think of a better examples but I'm sure they exist). Similarly we can use the "man" field of `package.json` to extract the man pages and lint them through "ManPageLintBear"
- Standardising: We may want to write the existing functionalities in terms of "Info Extractor" classes. For example, the functions to collect ignore globs from `.gitignore` file in `FileGlobs.py` can be converted into a `GitignoreInfoExtractor` class and then used through "Info Extraction" interface.
Moreover, `.npmignore` files can also be added to develop a more generic `IgnorePathsExtractor` (that's a question of design that needs to be discussed: if we want to limit an `InfoExtractor` class to one file or generalize them for multiple files but for a specific kind of information)
- Extracting information from various linter's configuration files as setting values for bears: As discussed by @jayvdb [here](https://gitter.im/coala/coala?at=59421d7af31c8ced0c394e33). A question of design is whether we want linter bears to use their external linter config file, or use settings from the .coafile. IMO, this is a feature for coala/coala rather than coala-quickstart.
- Another idea on similar lines for coala-quickstart is to impute setting values across the bears for these configuration files. For example, using the `indent_size=4` information in some linter's configuration file and using it for all the bears that have a setting `indent_size` (this will make more sense in terms of aspects approach).

More ideas for usage can be (as discussed by @jayvdb [here](https://github.com/coala/coala-quickstart/pull/102#issuecomment-307902931))

- Reusability across other coala repos.
- Using this information extraction framework for creating more complex info extraction tools or tools similar to quickstart.

It is arguable that these classes should be implemented in coalib from beginning only, but there are some reasons not to do that:
- Currently, the scope is clear just for coala-quickstart.
- There are many of third-parser libraries that would need to be installed for the Info Extraction, so the intention was not to bloat coalib unless required.
- As these classes are not much dependent on coala-quickstart, it may not be dificult to port them to coalib later.

Any ideas and suggestion regarding the design and scope of this "Inforamtion Extraction" framework? :sweat_smile:

status/needs-discussion

cc @adtac @jayvdb

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading cEP-0009 and the current Info Extractor implementation in coala-quickstart, then review the listed use cases and the existing ignore-glob functionality in FileGlobs.py. A useful outcome would be a resolved scope and placement decision for the framework, including whether it should remain in quickstart or move to coalib.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.