invoke-ai / invoke-ai/InvokeAI

[enhancement]: Rich text support in Web UI prompt field

Open
#2,583 2 comments 0 reactions 0 assignees View on GitHub
enhancement Inactive Issue prompting stale
Dominant language
Python
Stars
28.2k
Forks
3k
Avg merge
6d 5h
Merged PRs (30d)
19

Description

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Contact Details

_No response_

### What should this feature add?

Rich text support in the web UI's prompt field would enable several useful QoL improvements.
- Syntax highlighting could help to quickly distinguish prompt syntax and keywords from prompt tokens. It would also help to reveal simple typos with drastic effects, such as mismatched parentheses or quotation marks, and clarify the expected behavior in some edge cases:
- commas interpreted as either parameter delimiters or prompt tokens depending on presence of quotation marks
- hyphens and parentheses can be interpreted in unexpected ways. For example, these 3 sets of prompts give identical results within their set, but distinct results from each other. (`--log_tokenization` in the CLI makes their interpretation far clearer):
- `(hot-dog)` == `hot-dog` == `hot - dog`; *hyphen is treated as a token*
- `(hot- dog)` == `(hot)-dog` == `hot- dog` == `(hot)0.9dog`; *the hyphen is now attention weighting syntax*
- `(hot - dog)` == `\(hot-dog\) `; *the parentheses and hyphens are now both treated as tokens (possible bug?)*
- Prompt tokenization could be indicated by the hue of tokens in the prompt, as with `--log_tokenization` in the CLI.
- Attention weighting could be indicated by variations in brightness, saturation, or boldness (within the limitations of legibility).
- A floating panel above selected text could allow for easily increasing or decreasing attention weighting of the selected text, as with bolding or italicization in word processing applications.

As for implementation, something like [Slate](https://www.slatejs.org/) may be useful. It plays nicely with React and is what's used for Discord's text entry, but I haven't looked deeply into the full details yet.

### Alternatives

I often open a CLI instance of InvokeAI alongside my web session simply to use `--log_tokenization` for prompt debugging, but this has a severe impact on available VRAM.

### Aditional Content

Example UI as seen in Discord's text entry area:
![image](https://user-images.githubusercontent.com/308464/217473251-bad45f97-246b-403c-88ba-11ac2ec9f171.png)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.