gchq / gchq/CyberChef

"Bug" report: v10 rendering of UTF-8 characters > 255

Open
#1,627 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
JavaScript
Stars
35.8k
Forks
4.1k
Avg merge
2d 26m
Merged PRs (30d)
33

Description

**Describe the bug**
In the version 10 release notes under character encoding it states:
`The default value is 'Raw bytes' which attempts to treat every character as a byte in the range 0-255. If one of the characters has a Unicode value greater than 255, CyberChef will default to using UTF-8.`

Load a default instance of CyberChef, the input and output is set to Raw Bytes. When I enter a Unicode character > 255 the output pane is not defaulting to UTF-8 to display that data

Is the output pane expected to change encoding based on the input or am I misunderstanding the notes? Perhaps this only applies to the input pane

**To Reproduce**
Steps to reproduce the behaviour or a link to the recipe / input used to cause the bug:

Using this Unicode character as an example: https://www.compart.com/en/unicode/U+0421

1. Go to https://gchq.github.io/CyberChef/#input=0KE
2. Observe output pane

![image](https://github.com/gchq/CyberChef/assets/39072319/fec52515-8fa6-44e2-a1da-78a38eb5997d)

**Expected behaviour**
My reading of the release notes is that the output pane should switch to UTF-8 and the character displayed as designed.

![image](https://github.com/gchq/CyberChef/assets/39072319/d93cf66f-a117-445b-bc10-a9b154e76c16)

**Desktop (if relevant, please complete the following information):**
* Version: 10.5.2
* Compile time: 14/07/2023 18:03:18 UTC
* User-Agent:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36
* [Link to reproduce](https://gchq.github.io/CyberChef/#input=0KE&oenc=65001)

**Additional context**
Thanks as always.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the reported URL and comparing the input and output encoding controls with the version 10 release-note wording. Trace how the Raw Bytes setting handles the U+0421 example and determine whether the output should switch to UTF-8; done means the behavior matches the documented expectation or the documentation clearly explains the intended behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
frontend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.