ji-devs / ji-devs/ji-cloud

Text to Image

Open
#216 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

frontend - functionality needs more info
Dominant language
Rust
Stars
14
Forks
18
PR merge metrics
No merged PRs in 30d

Description

Currently, this is more of an R&D issue than a requirement to implement.

We have the following non-negotiable constraints on the horizon:

  1. Users must be able to have a WYSIWYG text editor with best-in-class Hebrew/English support. After testing several of the current editors on the market, it seems DraftJS meets this criteria - but conceptually this can be replaced by any text editor that has good UX and gives us some sort of raw data to work with programmatically.

  2. This text will sometimes be viewed as raw html (which all the editors support), but other times it must be rendered as an image. For example, in any module that requires chopping up the screen into pieces (puzzle, matching, etc.). It is also a good idea to plan for future capabilities - such as mixing with animation or in a 3d context. None of the editors support this out of the box.

  3. There is no reliable way to turn html into an image. It simply does not exist.

Therefore, we need to come up with a new solution. The following are some ideas, some of which we've tried before or have investigated, others are under-explored (perhaps for good reason):

  1. Using DraftJS and converting the data model to canvas text, manually (we had contracted someone to focus on that in the past. We got very good results at the time, but not perfect.)

  2. Similar, but instead of converting to canvas text, using Harffbuzz / FreeBidi / etc. - this is very low level and I'm worried it will be a lot of headache, especially to make it match 1:1, but it would be a good general solution and perhaps a welcome draftjs plugin too.

  3. html2canvas - buggy and unreliable (though perhaps our dev effort could be invested there to improve it for our use case)

  4. SVG foreignObject - I forgot the exact issue here, but testing in the past, it didn't work (I think it might have been an issue of loading web fonts from within the svg, as well as browser restrictions on security). Might be worth throwing a couple hours at it just to rule it out again, perhaps things have changed.

  5. DIY text input and only render (e.g. like step2 but not using any html text editot). Re-inventing Word doesn't sound like a small project

  6. Ignoring the problem altogether and using a serverside solution to open a browser, take a screenshot, and save/send that image. It's slow and potentially expensive. It's what we ended up doing on a previous project, and really sucks.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files, tests, or implementation entry points. Start by comparing the six listed approaches against the Hebrew/English editor, raw HTML, image rendering, animation, and 3D constraints; done means selecting and documenting a reliable direction rather than making a small isolated change.

Written by the indexing model from the issue text.

Assessment

Domain
frontend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.