SourceLink improvements for deterministic build of large git repositories
- Dominant language
- C#
- Stars
- 1.4k
- Forks
- 148
- PR merge metrics
- No merged PRs in 30d
Description
# Problem
Embedding current HEAD SHA to sourcelink.json means that any change in the repo affects the build outputs of all projects in the repo. That’s not ideal for reuse of build artifacts in deterministic build systems that are able to reuse previously built artifacts if their build inputs have not changed.
# Proposal
Git repository layout design allows for fast access to file content given its id (SHA). It also allows to quickly determine source file ids. Suppose we store object ids of all source files of a project in sourcelink.json along with a URL that returns the content of the file given an object id. Then the debugger could use that URL to look up content and sourcelink.json would no longer create artificial dependency among projects.
## Required web API
Currently, as far as I can tell, only VSTS git provides URL that returns raw content file given an object id:
`https://{account}.visualstudio.com/{project}/_apis/git/repositories/{repository}/blobs/{object-id}?api-version=4.1`
## SourceLink v2
Current version of SourceLink specification does allow to store object ids of all files but not efficiently.
For example,
```json
{
"documents": {
"/_/src/myproject/program.cs" : "https://account.visualstudio.com/project/_apis/git/repositories/repository/blobs/645b62889c921b3ba441d37dce9d02a02a9dd0df?api-version=4.1",
"/_/src/myproject/class.cs" : "https://account.visualstudio.com/project/_apis/git/repositories/repository/blobs/62889c921b3ba443ba441d37dce9d02a02a9dd0df?api-version=4.1"
...
}
}
```
The size is significant for large projects (~100KB for Microsoft.CodeAnalysis.dll).
We propose the following addition of _parameterized URLs_ to the SourceLink specification.
The names of the properties of "documents" object would remain unchanged - they would express path prefixes. Their values would either be of type `string` or `object`. The semantics of `string` values are unchanged. The `object` has the following properties:
### `url-format`: `string`
Interpolated string with holes with syntax `{N}`, where N is a zero-based index.
If `*` is present in this string it is substituted with the path suffix as in SourceLink V1. No more than one `*` is allowed.
### `params` : `object`
Encodes URL parameters.
Properties:
#### `encoding` : `string` (optional)
Currently supported encodings:
- `base64/deflate`
#### `values`: `string` | `object`
If encoding is not specified than `values` is of type `object` that maps document names or full paths to an array of values.
For example,
```
{
"a.cs": [ "645b62889c921b3ba441d37dce9d02a02a9dd0df", "Foo" ],
"fullpath1/b.cs": [ "549bfca89f672601e9632bbde2d0ed32cfa1b6b1", "Bar" ],
"fullpath2/b.cs": [ "40305ad59381a0cd6dfccd86745b619f4b5a51fe", "Jam" ]
}
```
If `encoding` is specified `values` is of type `string` and stores the values object encoded using the specified encoding (e.g. when encoding `base64/deflate` is specified the object is compressed and base64 encoded).
Example of SourceLink with object ids encoded in compressed parameters:
```json
{
"version": 2,
"documents": {
"/_/*": {
"url-format": "https://account.visualstudio.com/project/_apis/git/repositories/repository/blobs/{0}?api-version=4.1",
"params": {
"encoding": "base64/deflate",
"value": "lkjflskdjflskdjfls....dkjfklsdjf=="
}
}
}
}
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.