dotnet / dotnet/extensions

Text to Image: Additional operations

Open
#6,724 1 comment 0 reactions 1 assignee Claimed by @ericstj View on GitHub
area-ai
Dominant language
C#
Stars
3.2k
Forks
894
Avg merge
1d 12h
Merged PRs (30d)
23

Description

We should look at plumbing additional common operations and how those will work.

We've started out with text + image? + options. This handles generate, edit, variations, upscale.
- generate: text
- edit: text+image
- upscale: text=null + image + options.size != null
- variations: text=null + image + options.size == null

We don't yet have a happy path for the following operations which are somewhat common:
- Inpaint
- Outpaint
- Erase
- Find / Replace
- Multiple input images

Most can likely be achieved through RawRepresentationFactory but we shouldn't do that if they are common enough to have a first-class cross-provider experience.

References:
https://platform.openai.com/docs/api-reference/images
https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html
https://platform.stability.ai/docs/api-reference

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.