microsoft / microsoft/winml-cli

[Task] text-to-3d model support

Open
#292 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

model / task scale P2 triaged
Dominant language
Python
Stars
40
Forks
11
Avg merge
1d 8h
Merged PRs (30d)
50

Description

Overview

Text-to-3D models generate 3D geometry directly from a natural language description. TRELLIS supports text conditioning via the same SLAT pipeline as its image variant; shap-e (OpenAI) generates implicit neural representations from text; BrickGPT specializes in generating LEGO-like brick structures from text prompts.

Agent Scenarios

  • Creative content agent: generate 3D scene elements (characters, props, environments) from a designer's text description for prototyping
  • Game / metaverse asset agent: instantly produce game-ready geometry from narrative prompts without a 3D artist in the loop
  • Education / toy design agent: use BrickGPT to generate brick-by-brick building instructions from a text description of a structure
  • Concept visualization agent: turn a product brief into a rough 3D mockup for early-stage design review

ModelKit Integration

Models must pass the full wmk pipeline on all EPs:

wmk config → wmk build (ONNX export) → wmk perf → wmk eval

Acceptance Criteria

  • microsoft/TRELLIS-text-xlarge
  • microsoft/TRELLIS-text-large
  • openai/shap-e
  • AvaLovelace/BrickGPT

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the ModelKit integration flow described in the issue: run wmk config, wmk build for ONNX export, wmk perf, and wmk eval. Work through the four listed models—TRELLIS-text-xlarge, TRELLIS-text-large, shap-e, and BrickGPT—and consider the task complete when each passes the full wmk pipeline on all EPs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.