Design Meeting Notes (2023-10-23)

We had a pause - what happened?
- Vacations, explorations with internal teams (e.g. Copilot implementations), etc.
- Direct discussions with users, but took us away from GitHub for a bit.
Where are we now?
- Some TypeChat ideas prototyped in C# - https://github.com/microsoft/typechat.net
  - Partially to prove out integration with C# and Semantic Kernel.
  - Some long-term ideas prototyped - but want to come back to parity with the TypeScript implementation of TypeChat.
- TypeChat ideas also prototyped in Python - https://github.com/DanielRosenwasser/Pypechat
Still want more blog posts, want to have a video explainer - seeing is believing.
Plan to do a sweep over issues and PRs.

Things like Semantic Kernel, langchain, etc.
Currently exploring how these can be integrated - loose ideas at this moment?
Want to be able to find where these can complement each other, integrate better, etc.
- Planners based on TypeChat's JSON Programs

OpenAI functions are one function at a time.
Described via JSON schema.
There's a function role that fits within a conversation.
Fine-tuned - not guaranteed to get schema-conforming data (nor even well-formed data!).
Is there a lot of usage?
- There's a lot of excitement, but we haven't yet spoken with many users.
So why not just use the TypeChat approach here? Either TypeChat JSON validation or TypeChat JSON programs?
- We believe one subsumes the other - TypeChat being cross-model with type-checked validation is more robust.
- Could plug in your favorite schema validator to do this technically, right?
- Anecdotally, TypeChat performs very very well. To be honest, a lot better in our experience.
  - We're missing evidence we can show to the outside world though.
Do we have any insight into long-term plans with OpenAI functions?
- Not yet, we would love to discuss further with these teams.
Conclusion?
- We don't yet think it makes sense to support directly - would love to better understand long-term plans from LLM providers like OpenAI.

What's that mean?
- Verifiable and repairable syntactically/semantically
Areas of investigation
- Best representations of...
  - specifications (e.g. TypeScript types, "JSON templates", JSON schema...)
  - return formats (e.g. JSON, YAML, code in specific languages)
- Is there a compact schema form that we can adopt/invent with high accuracy? It'd be easier to verify if we had something more compact than JSON schema.
  - But new languages = new toolchains. Picking a well-defined subset of a known language like TypeScript might be more successful.
- How do we make these work across languages?
What about a separate authoring format?
- "SchemaLite"?
What about that subset of TypeScript?
What about TypeSpec?
TypeScript versus JSON Schema?
- TypeScript really shines on discriminated unions.
- What's the best way to describe a discriminated union to an LLM? For data interchange, that's fundamentally how you describe polymorphism.

Some feedback on programs is that they're cool, but too limited.
- Clever ways to enable some stuff like branching and iteration, but they don't always scale.
Models are being asked to produce an IR that is turned into another language, then interpreted.
The feedback loop from a type-checker is pretty removed.
- Hard problem with verification.
But we have concerns about sandboxing and guaranteed availability (i.e. keeping your host programs working in spite of the halting problem).
Plus, what if you have millions of functions, or methods on objects with thousands of types, etc.?
- And if we want to deliver plans with no hallucinations, we want to be able to summarize plans for humans too. So we want that...
- But how do you actually present this to a user?
- Just be able to provide transactions/undo? Commit/unroll?
Maybe there's some inspiration to be taken from languages like PowerShell, Tcl? Bring your own language features, build it up.

Dynamic Schema Generation from Data
Programmatic Schema Construction
- Dynamically populating structure and entities

Description