microsoft / microsoft/terminal

Loading a 96-character soft font will also replace a matching 94-character set

Open
#20,631 2 comments 0 reactions 0 assignees View on GitHub
Area-VT Issue-Bug Priority-3 Product-Conhost Product-Terminal
Dominant language
C++
Stars
105k
Forks
9.6k
Avg merge
3d 17h
Merged PRs (30d)
29

Description

### Windows Terminal version

1.25.1322.0

### Windows build number

10.0.19045.6332

### Other Software

[GIF320](https://github.com/jmason/gif320)

### Steps to reproduce

This is not something we should be fixing in any hurry (if at all), but I wanted to at least have a record of the issue, and why it is the way it is. Feel free to TLDR unless you're particularly interested in the subject.

The problem stems from the fact that you can have two character sets sharing the same ID, as long as they have different sizes. For example, British NRCS has an ID of `A`, but so does Latin-1. The difference is that British NRCS is a 94-character set, and Latin-1 is a 96-character set, and different set sizes are designated with different escape sequences.

We can designate British NRCS in G1, then select G1 as GL with a sequence like this:
```sh
echo -e '\e)A\x0Eabcd#\x0F'
```

The output should be `abcd£` (`£` vs `#` is the only difference from ASCII).

To do the same thing with Latin-1, we'd use `\e-A` rather than `\e)A` when designating the character set, so the equivalent sequence looks like this:
```sh
echo -e '\e-A\x0Eabcd#\x0F'
```

In this case the output should be `áâãä£`.

Now that we've seen the two character sets, let's try loading a 96-character soft font with the following sequence.
```sh
echo -e '\eP0;0;0;10;0;2;16;1{A~~~~~\e\\'
```

The last value in that parameter list sets the _Pcss_ option, which indicates the character set size: `1` means it's a 96-character set. The character following the `{` indicates the ID that we're replacing, so when you take into account that the size is 96, that `A` should represent Latin-1.

We obviously haven't got all 96 characters defined here, but this sequence is enough to get the font loaded - the missing glyphs will be rendered as reversed question marks. So once that's done, we can try looking at the two character sets again with the sequences we used before:
```sh
echo -e '\e)A\x0Eabcd#\x0F'
echo -e '\e-A\x0Eab#cd\x0F'
```

### Expected Behavior

You should still see `abcd£` when viewing the British NRCS set, but the Latin-1 set should now be rendered as `⸮⸮⸮⸮⸮`.

### Actual Behavior

Both British NRCS and Latin-1 have been replaced with the soft font, so they both show as `⸮⸮⸮⸮⸮`. That's technically a bug, but we're actually doing this deliberately, as explained in the comment here:
https://github.com/microsoft/terminal/blob/feee5fe6258e775590f7a3d12f1d65be2636dd4f/src/terminal/adapter/terminalOutput.cpp#L234-L236

The software that comment was referring to is the [GIF320 image viewer](https://github.com/jmason/gif320). It can convert a GIF into a 96-character soft font, which can then be output as a grid of characters to reproduce the image. But that utility has a bug: it mistakenly uses a 94-character designation sequence to load the 96-character font. It still works on a real VT320 though! You can see a photograph on their repo showing an actual DEC VT320 displaying one of the sample images.

So that's the reason why we currently allow that buggy designation sequence. But what I recently discovered was that the VT420 doesn't share that bug (and I expect the same would apply to the more modern terminals in the VT500 range).

The other thing I discovered (while attempting to port GIF320 to work on a VT420) was that they're actually sending _two_ `DECDLD` sequences when loading the character sets. One loads the font, but prior to that they send another `DECDLD` sequence that first clears the font buffer. There should be no need for that, but I noticed a comment in the code that says "ES_clear_soft_font is a bug workaround".

So it occurs to me now that the VT320 probably doesn't actually accept the incorrect designation sequence by default, and their code would initially not have worked. But they stumbled on this hack where sending an additional `DECDLD` reset was enough to trick the terminal into working (I'm assuming they weren't aware that 96-character sets are designated differently, so wouldn't have realised the bug was actually in their own code).

So why does the reset trick work? I _think_ it might be because the reset sequence is done with the _Pcss_ parameter set to `0`, implying that it's a 94-character set. And while it isn't actually defining a font at that size, it's perhaps still enough for the system to register the ID as being a 94-character set. So when they later re-register it as a 96-character set, I suspect the terminal forgets to clear that state, and is then happy to accept either designation sequence for selecting the font.

This is probably still a bug in the terminal, because that trick doesn't work on the VT420, but it's a far less significant bug than the one I initially thought they had (assuming my theory is correct). So we can probably fix our designation handling to address the problem I reported at the start of this issue, but still support software like GIF320 if we can emulate that secondary bug.

This is very much an edge case issue, though, so I don't think there's any need to do anything about it right now (especially since I'm still not sure of all the facts). But perhaps it's something we can try and solve in the future.

Contributor guide

Open the contributing guide

Research direction

Start with the soft-font designation handling and the ES_clear_soft_font workaround in src/terminal/adapter/terminalOutput.cpp, especially the linked comment. Reproduce the 94- and 96-character escape sequences, then compare the behavior with GIF320's reset sequence. Done means correct size-specific replacement without losing compatibility with the documented GIF320 behavior, supported by tests if an existing test entry point is found.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.