TypeInferenceProvider using another tool than pyre - Jedi
- Dominant language
- Python
- Stars
- 1.9k
- Forks
- 229
- PR merge metrics
- No merged PRs in 30d
Description
[What follows is probably heresy given that Pyre is another Instagram project, so please don't "throw me to the pyre" :fire: - I mean no offence to that project]
I've found setting up a working pyre environment somewhat painful (building watchman from source, then getting a core dump because my project path was too long, etc.) and the documentation in LibCST of how to actually setup a ``TypeInferenceProvider`` and ``FullRepoManager`` to be lacking an example. Indeed, the best I've found was a screenshot of a notebook in https://github.com/Instagram/LibCST/pull/179, which I've diligently transformed into indexable form below (for my future self who wants to be able to google an example of doing it):
```
import libcst
from libcst.metadata.full_repo_manager import FullRepoManager
from libcst.metadata.type_inference_provider import TypeInferenceProvider
m = FullRepoManager('libCST/', ['_maybe_sentinel.py'], {TypeInferenceProvider})
for node, type_str in m.get_metadata_wrapper_for_path('_maybe_sentinel.py').resolve(TypeInferenceProvider).items():
code = libcst.parse_module("").code_for_node(node)
print(f'{code}: {type_str}')
```
As a result, I've also looked at other means of getting inference data given a node... [``Jedi's Script.infer``](https://jedi.readthedocs.io/en/latest/docs/api.html#jedi.Script.infer) is an interesting and seemingly simple option despite Jedi and LibCST working on a different level (a Script in Jedi has the ability to look through a virtual environment / PYTHONPATH to find references etc., much like Pyre and the ``FullRepoManager`` concept in LibCST).
A quick prototype later, I have a means to get hold of the Jedi inference for a node through a metadata provider (I think this is a testament to the LibCST code that this is so simple to do :+1:):
```
import libcst as cst
import libcst.metadata
import jedi
from typing import List
class TypeInferenceFromJediProvider(cst.BatchableMetadataProvider[List[jedi.api.classes.Name]]):
METADATA_DEPENDENCIES = (libcst.metadata.PositionProvider, )
gen_cache = True # We need the cache to contain a jedi.Script.
def __init__(self, cache) -> None:
super().__init__(cache)
self._script: jedi.Script = self.cache['script']
def _parse_metadata(self, node: cst.CSTNode) -> None:
pos = self.get_metadata(libcst.metadata.PositionProvider, node).start
self.set_metadata(node, self._script.infer(pos.line, pos.column))
def visit_Name(self, node: cst.Name):
self._parse_metadata(node)
def visit_Attribute(self, node: cst.Attribute):
self._parse_metadata(node)
def visit_Call(self, node: cst.Call):
self._parse_metadata(node)
```
Which is used as:
```
class InferencePrinter(cst.CSTVisitor):
METADATA_DEPENDENCIES = (cst.metadata.PositionProvider, TypeInferenceFromJediProvider)
def visit_Name(self, node: cst.Name) -> None:
pos = self.get_metadata(cst.metadata.PositionProvider, node).start
possible_types: List[jedi.api.classes.Name] = self.get_metadata(TypeInferenceFromJediProvider, node)
print(f"{node.value} found at line {pos.line}, column {pos.column}")
print(f"Source: {', '.join(possible_type.full_name for possible_type in possible_types)}")
print()
prj = jedi.Project('example-project')
code = '''
from collections import namedtuple
class MyThing(namedtuple('MyThing', ['a'])):
pass
thing = MyThing()
thing.__len__
'''
script = jedi.Script(code, project=prj)
module = cst.parse_module(code)
wrapper = cst.metadata.MetadataWrapper(module, cache={TypeInferenceFromJediProvider: {'script': script}})
wrapper.visit(InferencePrinter())
```
Output:
```
collections found at line 2, column 5
Source: collections
namedtuple found at line 2, column 24
Source: collections.namedtuple
MyThing found at line 4, column 6
Source: __main__.MyThing
namedtuple found at line 4, column 14
Source: collections.namedtuple
thing found at line 7, column 0
Source: __main__.MyThing
MyThing found at line 7, column 8
Source: __main__.MyThing
thing found at line 9, column 0
Source: __main__.MyThing
__len__ found at line 9, column 6
Source: builtins.tuple.__len__
```
Given the knowledge that Jedi and LibCST are both using parso under the hood, I didn't look into trying to avoid multiple parsing stages (performance isn't so critical to me, and the performance was good enough).
I just wanted to write this down so that others can benefit - I don't expect this will make its way into the LibCST codebase. (please feel free to close the issue!)
Contributor guide
Assessment
This issue has not been assessed yet.