Performance Area: Environmental Variables

#### Overview

`System.Environment.GetEnvironmentVariable()` is something we call quite a bit, approximately 8 times per function invocation on Windows (this can vary by config and other environments). Consider 8 the low end, Linux calls it a few more times.

Adjusting just those 8 out (disregarding more on Linux and additional remaining calls in WebJobs SDK, e.g. for App Insights), we can get 2-3% gains, for example **going from ~47,300 request/sec to ~49,000 requests/sec** and slightly reduced latency. This is for 2 reasons:
1. The access isn't free (and may cost even more in prod DWAS environments...but it's almost certainly not _better_).
2. It turns out the implementation allocates on every call because [a new string is created each time](https://github.com/dotnet/runtime/blob/57bfe474518ab5b7cfe6bf7424a79ce3af9d6657/src/libraries/System.Private.CoreLib/src/System/Environment.Variables.Windows.cs#L13-L31), adding to our gen0 GC load.

Here's an example benchmark getting `"TEMP"` (33 bytes long, for a 66 byte string in this case):
```cs
public class EnvironmentTests
{
private EnvironmentCache PropCache;
private IDictionary Cached;

[GlobalSetup]
public void Setup()
{
var currentVars = Environment.GetEnvironmentVariables();
Cached = currentVars;
PropCache = new EnvironmentCache(currentVars);
}

[Benchmark]
public string GetEnvironmentVariable() => Environment.GetEnvironmentVariable("TEMP");

[Benchmark]
public string Dictionary() => Cached["TEMP"] as string;

[Benchmark]
public string Prop() => PropCache.TempDirectory;

[Benchmark]
public string PropLazy() => PropCache.TempDirectoryLazy;

public class EnvironmentCache
{
private readonly IDictionary Cached;
public EnvironmentCache(IDictionary variables)
{
Cached = variables;
TempDirectory = Get("TEMP");
}

public string TempDirectory { get; }

private string _tempDirectory;
public string TempDirectoryLazy => _tempDirectory ??= Get("TEMP");

public string Get(string variableName) => Cached[variableName] as string;
}
}
```
To compare the options here from direct access to cached as dictionary, to lazy prop cache, to explicit prop cache:

| Method | Mean | Error | StdDev | Gen 0 | Allocated |
|------------------------- |-----------:|----------:|----------:|-------:|----------:|
| GetEnvironmentalVariable | 58.0679 ns | 1.2067 ns | 1.8786 ns | 0.0052 | 88 B |
| Dictionary | 20.0036 ns | 0.4191 ns | 0.3921 ns | - | - |
| Prop | 0.1044 ns | 0.0330 ns | 0.0292 ns | - | - |
| PropLazy | 1.0480 ns | 0.0736 ns | 0.1382 ns | - | - |

Overall, we can improve the access time per property about ~58-580x. These aren't huge absolute values per call, but we're calling it a lot and the multiplier is large.

Example benchmarks before (current `dev` branch):
```lua
➜ .\bombard.exe -k -l -c 100 -d 300s "http://localhost:5002/api/HttpTriggerAnon?name=bob"
Bombarding http://localhost:5002/api/HttpTriggerAnon?name=bob for 5m0s using 100 connection(s)
Done!
Statistics Avg Stdev Max
Reqs/sec 47591.54 8274.01 59695.82
Latency 2.10ms 345.55us 296.07ms
Latency Distribution
50% 2.00ms
75% 2.00ms
90% 2.29ms
95% 3.00ms
99% 9.00ms
HTTP codes:
1xx - 0, 2xx - 14270905, 3xx - 0, 4xx - 0, 5xx - 0
others - 0
Throughput: 14.47MB/s
➜ .\bombard.exe -k -l -c 100 -d 300s "http://localhost:5002/api/HttpTriggerAnon?name=bob"
Bombarding http://localhost:5002/api/HttpTriggerAnon?name=bob for 5m0s using 100 connection(s)
Done!
Statistics Avg Stdev Max
Reqs/sec 47171.59 8344.88 59850.90
Latency 2.12ms 337.17us 103.73ms
Latency Distribution
50% 2.00ms
75% 2.00ms
90% 2.34ms
95% 3.00ms
99% 9.00ms
HTTP codes:
1xx - 0, 2xx - 14147124, 3xx - 0, 4xx - 0, 5xx - 0
others - 0
Throughput: 14.35MB/s
➜ .\bombard.exe -k -l -c 100 -d 300s "http://localhost:5002/api/HttpTriggerAnon?name=bob"
Bombarding http://localhost:5002/api/HttpTriggerAnon?name=bob for 5m0s using 100 connection(s)
Done!
Statistics Avg Stdev Max
Reqs/sec 47302.86 8405.61 65003.25
Latency 2.11ms 327.15us 111.18ms
Latency Distribution
50% 2.00ms
75% 2.00ms
90% 2.28ms
95% 3.00ms
99% 9.00ms
HTTP codes:
1xx - 0, 2xx - 14186460, 3xx - 0, 4xx - 0, 5xx - 0
others - 0
Throughput: 14.39MB/s
```
And after:
```lua
➜ .\bombard.exe -k -l -c 100 -d 300s "http://localhost:5002/api/HttpTriggerAnon?name=bob"
Bombarding http://localhost:5002/api/HttpTriggerAnon?name=bob for 5m0s using 100 connection(s)
Done!
Statistics Avg Stdev Max
Reqs/sec 49064.01 8487.65 62154.62
Latency 2.04ms 335.66us 120.45ms
Latency Distribution
50% 2.00ms
75% 2.00ms
90% 2.00ms
95% 2.82ms
99% 8.64ms
HTTP codes:
1xx - 0, 2xx - 14715195, 3xx - 0, 4xx - 0, 5xx - 0
others - 0
Throughput: 14.92MB/s
➜ .\bombard.exe -k -l -c 100 -d 300s "http://localhost:5002/api/HttpTriggerAnon?name=bob"
Bombarding http://localhost:5002/api/HttpTriggerAnon?name=bob for 5m0s using 100 connection(s)
Done!
Statistics Avg Stdev Max
Reqs/sec 49114.93 8393.16 60212.43
Latency 2.03ms 330.49us 89.50ms
Latency Distribution
50% 2.00ms
75% 2.00ms
90% 2.00ms
95% 2.77ms
99% 8.62ms
HTTP codes:
1xx - 0, 2xx - 14731183, 3xx - 0, 4xx - 0, 5xx - 0
others - 0
Throughput: 14.94MB/s
➜ .\bombard.exe -k -l -c 100 -d 300s "http://localhost:5002/api/HttpTriggerAnon?name=bob"
Bombarding http://localhost:5002/api/HttpTriggerAnon?name=bob for 5m0s using 100 connection(s)
Done!
Statistics Avg Stdev Max
Reqs/sec 48831.45 8393.89 64500.00
Latency 2.05ms 317.95us 120.81ms
Latency Distribution
50% 2.00ms
75% 2.00ms
90% 2.00ms
95% 2.95ms
99% 9.00ms
HTTP codes:
1xx - 0, 2xx - 14646744, 3xx - 0, 4xx - 0, 5xx - 0
others - 0
Throughput: 14.85MB/s
````

#### Proposal

I have a branch to demonstrate the wins of caching in `craver/environment-cache` ([comparison link](https://github.com/Azure/azure-functions-host/compare/craver/environment-cache?expand=1)).
Note: Please ignore the `IEvironment` being a class, that should be renamed and I was avoiding massive diff from a rename for the purposes here.

This implementation caches environmental variables on `SystemEnvironment` (really down in the base class). It does NOT flush the cache when the variables change - that's what we need to see if we can reliably do on Linux and DWAS (when needed). There are a few things this brings to the table:
1. Performance: 2-3% gains in the Functions host (this does *not* include additional wins in the WebJobs SDK, which accesses more variables each request - we'd see additional wins from the same there especially in the Application Insights path).
2. Usability: consistent access to environmental variables if taken to completion, e.g. instead of using `EnvironmentSettingNames` everywhere, we'd have properties added to the ones in this proposal. Each of those properties can have a usable description.
3. Simplicity: we can likely move a chunk of `EnvironmentExtensions` to properties here - they're already extensions on IEnvironment so the object hierarchy doesn't change, they can just be more efficiently cached properties.

Overall, this aligns to configuration akin to `IOptions`. We could make the accessors in the base take into account `IOptions` and config files for consistent access everywhere down the road. In all, this is object oriented access to settings in a very optimized way.

What's needed is figuring out the flush. For example normally on Windows we'd trigger [the `SystemEvents.UserPreferenceChanged` event](https://docs.microsoft.com/en-us/dotnet/api/microsoft.win32.systemevents.userpreferencechanged?redirectedfrom=MSDN&view=dotnet-plat-ext-6.0). The [sample branch](https://github.com/Azure/azure-functions-host/compare/craver/environment-cache?expand=1) has a method `SystemEnvironment.Recache()`. If we can identify an event when we need to recache, this is a viable strategy, but I'm not familiar enough with DWAS and Linux hosting to be sure - so writing this up to ask the teams :)

#### Open Questions
1. Do we have a system event, or can we add one to trigger the recache? (when specialization occurs?)
2. If we have an event and this is instant, can we use the same event to save 0-500ms on [the specialization loop](https://github.com/Azure/azure-functions-host/blob/f1c6016b8e3a0f6ad8f02eb04297badf118e7acb/src/WebJobs.Script.WebHost/Standby/StandbyManager.cs#L214)? (cc @mathewc)

Even in the case where we get an HTTP request to trigger specialization, it's possible we can shave slightly with an event-based approach here. We'd do so by complicating the recache method slightly to keep an old/current comparison of needed variables to see if we just tripped for specialization.

cc @fabiocav @brettsam