.NET Core paket and dotnet/sdk

Paket depends now on .NET framework (win) and mono (unix/mac).

This is really good because the normal paket.exe console app can run xplat if mono/.net is installed.

For some scenario, like .net core development (or fable) with new dotnet/sdk (new csproj/fsproj), the mono/.net shouldnt be needed.

To remove that deps of paket in these scenario there are some points to address:

• xplat paket.exe console app, based on .net core
• boostrapping process (from clean repo)
  • minimal set of files (and size) needed to be committed on repo
  • self updating of boostrapping process
  • binaries sources: nuget/github
  • chicken/egg. how to boostrap the boostrapper
• execute paket from command line, xplat. with good dev UX
  • magic mode
• auto-update of paket
  • frequent release schedule -> minimize size/download
  • binaries sources: nuget/github
• minimize UX changes for users
• maintenance.
  • try to avoid two version (net/netcore) because old sdk project can also use .net core based paket
• tradeoff downloadsize vs dependencies

Atm ihmo boostrapping and auto-updates are the real issue

Current Plan

• make paket console app to .net core so works the same as now if published as SCD (wip #2918)
• implement dotnetcli tools, per repo ( ref #2919 )
• modify build.fsx to put in Paket nuget both paket .net core FDD (in tools/netcoreapp2.0) and paket .net fw (in tools/net451)
• reuse paket SCD itself as boostrapper, or use a dotnetclitool, to add paket in PATH

Notes

notes about console deployment options on .net core

Options are:

• SCD, self contained deployment
  • bigger, one release for platform (unix/win/macosx), require just .net core runtime 2.0 DEPS installed
• FDD, framework dependent deployment
  • smaller size
  • require .net core 2.0 runtime installed
  • execution is dotnet paket.dll, so require a wrapper script to execute for UX (paket.cmd/paket.sh)
  • where is dotnet? is the right version installed?
• DotnetCliTool
  • just a proj (few lines of text) is required
  • update based on just nuget.org
  • require restore
  • require .net core sdk installed
  • run as dotnet paket
  • require execution from same dir of proj, so need wrapper scripts (paket.cmd/paket.sh)

File size:

	DotnetCliTool	FDD	SCD
File Size	0,1 MB	1MB	60 MB
Requirements	.net sdk + runtime + runtime deps (120MB)	.net core runtime + runtime deps (60mb)	.net core runtime deps (native pkg)

notes about paket, dotnet/sdk, .net core

paket is a strange tool, because atm for restore process, need to be there BEFORE the restore.
or at least a target file to override default nuget implementation in dotnet/sdk
instead old sdk require the exe to restore/update projs

But lots of challenges are shared with normal tools in .net ecosystem

There is no builtin per repo tools. or a way to script (like an fsx).
AFAIK the most similar to per repo tools are the cli tools or the FDD.
AFAIK the most similar to script is an msbuild target + optional task in assembly

How now .net sdk support external tools? or custom execute code xplat.

• Dotnet Cli tools (per project)
• msbuild targets
• built it on machine. dotnet new + override Program.cs + dotnet publish

Other package manager expect the dev sdk installed, and use that to boostrap:

• npm -> node
• pip -> https://bootstrap.pypa.io/get-pip.py -> python
• bundler -> gems -> ruby

Possible solutions

My current ideas, not in preference order.

plan A

support tools at repo level natively in dotnet/sdk. ref https://github.com/dotnet/cli/issues/4723

PRO:

• native support in dotnet/sdk
• less maintenance by paket team
• no boostrapper needed. the sdk is the boostrapper.

CONS:

• native support in dotnet/sdk
  • require new version of dotnet/sdk. probably at least 2.2 or 3.x
  • will not work on older VS version or just msbuild (but VS now bundle .net core sdk...)
• depends on dotnet/sdk team
  • slower to patch/workaround, based on .net team schedule not paket team
• require .net sdk installed, but native paket app can be download if needed, that's just for boostrap
• cannot replace current paket for older sdk

plan B

Why: paket deps doesnt change a lot. paket does.

like now, using native app (SCD) but smarter update mechanism

• paket published as SCD and FDD
• current paket.boostrapper.exe as SCD
• SCD are big, need a smart update mechanism
  1. first usage download latest native console binaries (per OS)
  2. updating is just xplat paket FDD, so paket.dll and non-bcl managed deps (paket.core.dll, etc)
  3. if not possible the minimal update, do step 1
• boostrapping process start with a dotnet msbuild (target + dll), to just download paket.boostrapper.exe

PRO:

• pretty much like now
• can replace current boostrapping
  • new deps will be just .net core 2 runtime deps, not mono/.net
• boostrapping can also download the net version like now, if want to support that too

CONS:

• maintenance on paket team
• paket specific :( a missed opportunity
• download paket native boostrapper is big. but can be cached.

plan C

Why: support and enhance .net cli tools. use that support for paket itself.

Add support of repo level tools to paket and .net cli tools. Use a built paket native (SCD) to boostrap paket (replace previous paket.boostrapper.exe).
The chicked/egg solution.

• paket as packaged console app (like a .net cli tool, ex dotnet-fable), so small and easy to pack/version
• boostrapper (paket native):
  • is paket, with a paket.depencies who contains paket-cli as tool
• add .net cli tools support to paket
  • the .net cli tool (like dotnet fable) just invoke dotnet with args and a file with deps. paket generate these args/files needed
  • generate a shims/script wrapper, so a dotnet hello tool can be invoked as hello. and paket as paket
• boostrapping based on dotnet msbuild to just download native paket per OS

PRO:

• cli tools can be in also used paket.deps/lock. per repo.
• boostrapper is not a special case, is just paket built as native console app
• minimal size.
• enable other tools (good). like dotnet fable as fable. will work in whole repo not just the proj dir.

CONS:

• download paket native boostrapper is big. but can be cached.
• boostrapper paket schedule <> paket schedule, so doesnt need to update often. more maintance.
• atm cli tools are from nuget feed. github fallback need to be implemented (paket download nupkg from github to a dir -> add that dir as feed -> restore)

Conclusions

• plan A: .net oss way, but slower to implement and require at least .net sdk vNext version as deps.
• plan B: upgrade current boostrapping mechanism
• plan C: remove current bootstrapper, replace with paket who support .net cli tools

Atm i prefer plan C, while is longer than B to implement, add a lot more value with less maintenance long term.
Plan A is ok, but doesnt help paket in medium term, because cannot replace current way.

TODO

as a note, i volounteer to implement the choosen solution. if someone want to help, will be nice, because tasks can be split

Start by reading the Current Plan, build.fsx notes, and the referenced work in #2918 and #2919. Compare the proposed bootstrapper, .NET Core packaging, CLI-tool, and auto-update paths before choosing a bounded implementation. Done would require a decided approach that addresses the listed cross-platform bootstrap and update requirements.