microsoft / microsoft/aspire

Docker Model Runner (DMR) Support

Open
#11,434 0 comments 1 reaction 0 assignees View on GitHub
area-app-model
Dominant language
C#
Stars
6.3k
Forks
991
Avg merge
2d 15h
Merged PRs (30d)
196

Description

### Is there an existing issue for this?

- [x] I have searched the existing issues

### Is your feature request related to a problem? Please describe the problem.

It would be great if we could run models on DMR from Aspire, and then map those to either Azure or the equivalent docker compose or Kubernetes instances of the same.

### Describe the solution you'd like

Aspire Host for DMR with extensions to map to publication that uses the native docker model functionality locally and then will deploy automatically to the target of your choice mapping things like Azure models etc. and also into Kubernetes and docker compose. On docker compose it would use the native build up syntax for the models, and for Kubernetes it would need to do the equivalent deployment of some sort

### Additional context

This becomes an elegant solution for local dev because you can have multiple requests going against the tooling that is already on the system and it maps directly to docker compose and could be a great proxy to Azure etc. while getting native GPU easily etc.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.