DrKain / DrKain/subclean

[Feature Request] Support for other character encodings

Open
#8 1 comment 0 reactions 0 assignees View on GitHub
enhancement priotiry: low
Dominant language
TypeScript
Stars
64
Forks
6
PR merge metrics
No merged PRs in 30d

Description

Right now the tool will fail when trying to parse files with this character encoding. For a viable solution the tool should be able to detect the character encoding and convert to UTF-8 when required.
The converted data should be written even if nodes were not modified, this will remove the need to convert a file multiple times when running subclean on an entire library as a scheduled task.

See this https://github.com/DrKain/subclean/issues/7#issuecomment-948572760 for information on a temporary solution for the current problem.

Unfortunately this will require a dependency like [utf8](https://www.npmjs.com/package/utf8).

Test files:
- UCS-2 BE BOM: [subtitle.zip](https://github.com/DrKain/subclean/files/7389102/subtitle.zip)
- UTF-8-BOM: [subtitle.zip](https://github.com/DrKain/subclean/files/7387304/subtitle.zip)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the temporary solution in issue #7's linked comment and inspect how the CLI parses subtitle files and writes converted data. Use the UCS-2 BE BOM and UTF-8-BOM subtitle.zip fixtures to verify encoding detection, UTF-8 conversion, and writing output when nodes are unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
nodejs, typescript
Domain
cli
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.