System.Text.EncoderFallbackException: Unable to translate Unicode character \uD83D at index 14 to specified code page.
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- csharp, powershell, vscode
- Domain
- cli, developer-experience
Research direction
Reproduce the issue in the VS Code integrated PowerShell terminal using the filename and steps provided. Start with the PSReadLine history path shown in the exception, especially MaybeAddToHistory and WriteHistoryRange, and compare encoding behavior with the working Windows Terminal case. Done means Unicode paste and execution complete without an exception or character corruption.
Written by the indexing model from the issue text.
Description
Prerequisites
- Write a descriptive title.
- Make sure you are able to repro it on the latest released version
- Search the existing issues, especially the pinned issues.
Exception report
Oops, something went wrong. Please report this bug with the details below.
Report on GitHub: https://github.com/PowerShell/PSReadLine/issues/new
### Environment
PSReadLine: 2.0.0
PowerShell: 5.1.22621.6133
OS: Microsoft Windows 10.0.22631
Last 134 Keys
m a r k d Tab Spacebar F i n Tab Backspace Backspace Backspace . \ Spacebar F i n d i n g Spacebar E m u l a t o r . m d " Ctrl+LeftArrow Ctrl+LeftArrow Ctrl+LeftArrow Ctrl+LeftArrow Ctrl+RightArrow LeftArrow LeftArrow LeftArrow " Enter
UpArrow Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace l s Enter
� Spacebar F i n d i n g Spacebar E m u l a t o r . m d " Home " Home m a r k d o w n l i n t Spacebar Enter
### Exception
System.Text.EncoderFallbackException: Unable to translate Unicode character \uD83D at index 14 to specified code page.
at System.Text.EncoderExceptionFallbackBuffer.Fallback(Char charUnknown, Int32 index)
at System.Text.EncoderFallbackBuffer.InternalFallback(Char ch, Char*& chars)
at System.Text.UTF8Encoding.GetBytes(Char* chars, Int32 charCount, Byte* bytes, Int32 byteCount, EncoderNLS baseEncoder)
at System.Text.EncoderNLS.GetBytes(Char[] chars, Int32 charIndex, Int32 charCount, Byte[] bytes, Int32 byteIndex, Boolean flush)
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
at System.IO.StreamWriter.Dispose(Boolean disposing)
at System.IO.TextWriter.Dispose()
at Microsoft.PowerShell.PSConsoleReadLine.<>c__DisplayClass83_0.<WriteHistoryRange>b__0()
at Microsoft.PowerShell.PSConsoleReadLine.WithHistoryFileMutexDo(Int32 timeout, Action action)
at Microsoft.PowerShell.PSConsoleReadLine.MaybeAddToHistory(String result, List`1 edits, Int32 undoEditIndex, Boolean fromDifferentSession, Boolean fromInitialRead)
at Microsoft.PowerShell.PSConsoleReadLine.InputLoop()
at Microsoft.PowerShell.PSConsoleReadLine.ReadLine(Runspace runspace, EngineIntrinsics engineIntrinsics, CancellationToken cancellationToken)
Screenshot
VSCode's folder view displays name correctly
VSCode integrated PowerShell command (auto-complete)
Filename is not displayed correctly, exception occurred when executing the command:
Outside PowerShell has no problem/issue
The issue only occurs when running PowerShell which is integrated to VS Code, "normal" PowerShell executed from File Explorer works even with the UNICODE character 🕮:
Environment data
### Windows Terminal / PowerShell (the working one)
PS Version: 5.1.22621.6133
PS HostName: ConsoleHost (Windows Terminal)
PSReadLine Version: 2.0.0
PSReadLine EditMode: Windows
OS: 10.0.22621.1 (WinBuild.160101.0800)
BufferWidth: 120
BufferHeight: 30
### VS Code's integrated terminal (the buggy one)
PS Version: 5.1.22621.6133
PS HostName: ConsoleHost (Windows Terminal)
PSReadLine Version: 2.0.0
PSReadLine EditMode: Windows
OS: 10.0.22621.1 (WinBuild.160101.0800)
BufferWidth: 120
BufferHeight: 30
Since they're the same the issue seems to be in the integration of PS into VS Code.
Steps to reproduce
- Open VS Code
- Select/mark file (F2, Ctrl+A, Ctrl+C)
🕮 Finding Emulator.mdin file tree view - Run
🕮 Finding Emulator.md(INFO: Here the filename is correct but in Terminal window it looks like:
Expected behavior
No exception: Copy&Paste should work with terminal window / VS Code; looks like they're using different encoding
Workaround: Type markdownlint and press until ‘🕮 Finding Emulator.md’ appears. If the folder contains many files, you’ll need multiple tab presses because VS Code’s tree view sorts differently than the terminal’s completion order.
Actual behavior
Copy&Paste inserts wrong UNICODE char + when executing the exception occurs.
- Dominant language
- C#
- Stars
- 4.4k
- Forks
- 341
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from PowerShell/PSReadLine
-
Needs-Triage :mag:
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
PowerShell/PSReadLine#5205 ·
-
Needs-Triage :mag:
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
PowerShell/PSReadLine#5195 ·
-
Needs-Triage :mag:
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
PowerShell/PSReadLine#5121 ·
-
Needs-Triage :mag:
Difficulty 2/5 1-3 hours Newbie friendliness 64/100
PowerShell/PSReadLine#5045 ·
-
Area-CommandHelp Issue-Enhancement
Difficulty 1/5 Under an hour Newbie friendliness 68/100
PowerShell/PSReadLine#3470 · 3 reactions ·
All issues in PowerShell/PSReadLine
Similar issues
-
bug
Difficulty 1/5 Under an hour Newbie friendliness 75/100
sillsdev/languageforge-lexbox#2665 ·
-
bug documentation frontend
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
azurenoops/spin_agent#975 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
-
:watch: Not Triaged 11.0 fundamentals/subsvc
Difficulty 2/5 1-3 hours Newbie friendliness 92/100
dotnet/AspNetCore.Docs#37699 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
SubtitleEdit/subtitleedit#15108 · 1 comment ·