microsoft / microsoft/TRELLIS.2

Image to 3D textured object

Open
#48 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
11.3k
Forks
1.4k
PR merge metrics
No merged PRs in 30d

Description

It would be nice if you could load both pipelines

# Load image to 3D Pipeline
iTo3d_pipeline = Trellis2ImageTo3DPipeline.from_pretrained("microsoft/TRELLIS.2-4B")
iTo3d_pipeline.cuda()

# Load texture Pipeline
texture_pipeline = Trellis2TexturingPipeline.from_pretrained("microsoft/TRELLIS.2-4B", config_file="texturing_pipeline.json")
texture_pipeline.cuda()

Then simply run something like:

with timer(f"getting mesh for: {image_name}"):
    image = Image.open(image_file)
    mesh = iTo3d_pipeline.run(image)[0]
    mesh.simplify(MAX_FACES) #
  
  with timer(f"adding texture to mesh for: {image_name}"):
    image = Image.open(image_file)
    output = texture_pipeline.run(mesh, image)
    output.export(glb_outfile, extension_webp=True)

But the mesh output from Trellis2ImageTo3DPipeline, cannot be used directly in Trellis2TexturingPipeline.

There must a simple conversion though?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Trellis2ImageTo3DPipeline and Trellis2TexturingPipeline entry points shown in the issue, and reproduce the example using the two pipelines. Inspect the mesh returned by the image-to-3D pipeline and the input expected by the texturing pipeline. Done means the generated mesh can be passed through texturing and exported as a GLB.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-graphics, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.