Language Extensibility

## GOALS
Primary Goal
- Enable adding new languages/frameworks in an extensible way
Secondary Goals
- Run Functions in a separate process from the host
- More granular metrics from process
- More granular control over function lifetime
- Allow better control of frameworks (i.e. Node.js version, dotnet core)

## SCENARIOS

### BASIC JAVA FUNCTION

A customer needs to run a Functions implemented in Java.
1. They upload a .jar and function.json into a given Function directory.
2. The function.json contains a timer trigger and a queue output binding.
3. The function.json does not contain a scriptFile or entryPoint setting.
4. The Java based Function implemented an SDK provided by a built-in Java host.
When their function is triggered, the host invokes the Function via a built-in Java host environment. The function produces an output for the queue, which the built-in Java host provides the script host.

### BASIC NODE.JS FUNCTION

A customer needs to run a Function implemented in JavaScript or which has been transpiled to JavaScript
1. They upload a .js file and a function.json to a given Function directory.
2. The function.json contains a just trigger and binding data. It does NOT contain scriptFile or entryPoint.
3. The .js file exports a single method on an object or provides a default export.

- [ ] WIP

- [ ] Need more scenarios

## DEFINITIONS

1. Runtime/Host – Azure WebJobs SDK Script Host
2. Language worker/worker – The external process running functions in various languages/frameworks
3. Message broker – Handles communication between the “host” and the “language worker”

## REQUIREMENTS

### HIGH LEVEL REQUIREMENTS

| Category | Priority | Requirement | Notes |
| - | - | - | - |
| Host | 0 | Host must be able to send messages to the language worker triggering executions|
| Host | 0 | Host must be able to manage the process hosting the language worker.|
| Host | 1 | Host should use a mutli-framework/language IDL/API to define message types Recommend using ProtoBuf |
| Host | 1 | Host should use a securable protocol with implementations available in a variety of languages Recommend using gRPC |
| Host | 1 | Host should be able send data from input bindings (including trigger) to the language worker |
| Host | 1 | Host should be able to receive data from output bindings from the language worker |
| Host | 1 | Host should be able to receive streaming logs from the language worker |
| Host | 1 | Host should provide ability to have “built-in” language workers for various frameworks which load a customer’s Functions from framework specific files |
| Host | 2 | Host should provide ability to specify a custom language worker |
| Host | 3 | Host should be able to provide graceful warning to the language worker that there is a shutdown |
| Host | 3 | Host should be able to request a timeout for a given function running on a language worker
| Host | 3 | The host should be able to receive an ACK that the Function has been terminated safely.
| Host | 3 | If the language worker does not ACK a safe shutdown, the host should perform a shutdown of the language worker. |
| Portal | 1 | Portal should handle scriptFile pointing at a file not in a given Functions directory |
| Portal | 1 | Portal should handle binary/non-human readable content gracefully |

## MESSAGE TYPES

- [ ] TBD

## SUPPORTED DATA TYPES FOR BINDINGS

- [ ] TBD

## BINDING METADATA

- [ ] TBD

## INVOCATION LIFECYCLE MANAGEMENT

### START

1. The host should communicate with the language worker to invoke the Function
2. The language worker should start a function execution when it received a new invocation message

### COMPLETE

- [ ] TBD

### ERROR

- [ ] TBD

### TIMEOUT

1. A language worker should handle Function timeout/cancelation messages
2. A language worker should ACK if the Function gracefully timed out based on a configurable grace period
3. A language worker should provide a means for a Function to be notified of graceful shutdown
4. The host should send the timeout/cancelation message for a given Function execution to the language worker processing it
5. A language worker should wait for an ACK for a configurable grace period
6. The grace period should be configurable via the host.json (timeoutGracePeriod)
7. The grace period can be overridden per Function via function.json (timeoutGracePeriod)
8. The default grace period should be 1 second

## FUNCTION LIFECYCLE MANAGEMENT

### LOAD

- [ ] TBD

### UNLOAD

- [ ] TBD

## FUNCTION LOGGING

- [ ] TBD

## LANGUAGE WORKER LIFECYCLE MANAGEMENT

- [ ] TBD

## LANGUAGE WORKER RESOLUTION

- [ ] TBD

## LANGUAGE WORKER LOGGING

- [ ] TBD

### LOGGING PIPELINES

Categories:
1. Message broker
2. Stderr
3. Crash dump to a logs directory

### EVENTS

- [ ] TBD

### CATASTROPHIC FAILURE LOGGING

1. Log to $env:HOME/logFiles/… /functions/worker/{name}/
- Possibly this is a startup argument as well (makes CLI easy)

## HOST LOGGING

- [ ] TBD

### EVENTS

- [ ] TBD

Start with the Host requirements and the language worker, message broker, invocation lifecycle, and resolution sections. The issue names no source files or tests; a complete effort would need agreed message types, binding data, lifecycle behavior, and logging requirements before implementation can be considered done.

Written by the indexing model from the issue text.