google / google/haskell-indexer

Folding sift into haskell-indexer

Open
#79 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Haskell
Stars
103
Forks
19
PR merge metrics
No merged PRs in 30d

Description

Hi,

We're currently reviewing haskell-indexer for use on auditing work. I wrote a tool called [sift](https://github.com/fpco/sift) which is capable of generating a simple cross-package call graph of a haskell package and writing it to a .json file. Example:

```haskell
$ sift trace sift-bindings/*/* --flag-binding "ghc-prim GHC.Prim raise#" --call-trace | head -n 30
Flagged binding: ghc-prim:GHC.Prim.raise#
Used by aeson:Data.Aeson.Encoding.Builder.day
Call trace:
aeson:Data.Aeson.Encoding.Builder.day
|
+- base:GHC.Real.quotRem
| |
| +- base:GHC.Real.divZeroError
| | |
| | `- ghc-prim:GHC.Prim.raise#
| |
| `- base:GHC.Real.overflowError
|
`- base:GHC.Err.error

Used by aeson:Data.Aeson.Encoding.Builder.digit
Call trace:
aeson:Data.Aeson.Encoding.Builder.digit
|
`- base:GHC.Char.chr
|
`- base:GHC.Err.errorWithoutStackTrace
|
`- base:GHC.Err.error
|
`- ghc-prim:GHC.Prim.raise#
```

Preferably, we'd like to use haskell-indexer to achieve the same thing instead of maintaining two codebases.

I think I can get the same info from [`TickReference` and `Tick`](https://github.com/google/haskell-indexer/blob/12d6a46de5c19be708aff5bd0ff4bf63a488eada/haskell-indexer-translate/src/Language/Haskell/Indexer/Translate.hs#L106..L122) and [XRef](https://github.com/google/haskell-indexer/blob/12d6a46de5c19be708aff5bd0ff4bf63a488eada/haskell-indexer-translate/src/Language/Haskell/Indexer/Translate.hs#L68) gives me a list of `TickReferences` and also a list of `Relation`.

I think from there I can produce a graph with `Data.Graph` by producing a list `[(nodeid,node,[nodeid])]` where the latter list is "my dependencies", like I do [here](https://github.com/fpco/sift/blob/2055cd78b1258a9b4b4b4a049fd63c41373ec914/sift/src/Sift.hs#L71..L78)

```haskell
-- | Graph all package bindings.
graphBindings ::
Set Binding
-> OrdGraph BindingId Binding
graphBindings bs =
ordGraph (map
(\binding -> (binding, bindingId binding, bindingRefs binding))
(Set.toList bs))
```
then I can produce a simple call graph like this:

```haskell
callTrace :: OrdGraph BindingId node -> Graph.Vertex -> Graph.Vertex -> [Tree [Char]]
callTrace g start flagged =
fmap
(fmap
(\v' ->
let (_, bid', _) = ordGraphVertexToNode g v'
in S8.unpack (prettyBindingId bid')))
(filterForest
(flip (Graph.path (ordGraphGraph g)) flagged)
(Graph.dfs (ordGraphGraph g) [start]))
```

So I'm 90% confident I can fairly readily get the information I need to obsolete the `sift` tool. I have some questions:

1. Do you have any intention of penetrating class instances so that we could, for example, determine that `throw#` is used by `read "x" :: Int` because the method instance for `Int` uses `error`? We need this for auditing, aside from it being a super cool feature in general.
2. It seems that there isn't support for `base`? What's your approach on that? We need this for auditing. Here's how `sift` tackles the tooling issues:
* sift-frontend-plugin is used when building GHC's `lib` dir, on the `base` package specifically. You just inject `--frontend` in the right place after building `sift-frontend-plugin` in the same package set. That lets you generate a profile of base. If you had trouble with this on haskell-indexer, maybe I can help out getting that to work.
* sift-compiler - this is a copy/paste of the commandline interface from intero and ghci, and on start it immediately generates a profile of all the modules. This lets you do `stack ghci --with-ghc sift-compiler` and then you're done. I believe `cabal repl --with-ghc sift-compiler` would also work, but I haven't tested it. Why not just `stack ghci --with-ghc ghci --ghci-options '--frontend Sift.FrontendPlugin'`? Because GHC rejects frontend plugins when used with `--interactive`. :man_shrugging:
3. Finally, would you accept a PR that would contain some code for the above code tracking? It'd be nice to fold my code into haskell-indexer, and maybe have a UI that could display a call graph interactively.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.