Formatting function to ASCIIfy punctuation only
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 2.1k
- Avg merge
- 4d 21h
- Merged PRs (30d)
- 31
Description
I've been using beets for a couple of years now and I love it. There's a minor annoyance for me that I've noticed since the beginning and have more or less ignored, but I thought I'd finally ask if there's anything I can do about it. Apologies if I've missed an existing solution in the config guide or setup.
### Problem
When beets imported my library it Unicode-ified a lot of previously plaintext ASCII tags & filenames. For example, "El-P - I'll Sleep When You're Dead" becomes "El‐P - I’ll Sleep When You’re Dead" (in both files & tags.)
These look almost the same, but the punctuation is Unicode-ified:
* The dash which was U+002D (ASCII "hyphen-minus") is now U+2010 ("hyphen")
* The apostrophe U+0027 is now a right single quotation mark U+2019.
This isn't beets' doing, if I [download the JSON results for the musicbrainz link](https://musicbrainz.org/release/9652a87e-294a-4b37-a440-2a70c1cc3e1c?fmt=json) then these UTF-8 characters are used there.
Similar things apply for other punctuation marks, this is just a good example as it has two of them. :)
The annoyance is:
* Not all players can render UTF-8 tags properly (Kodi on Android seems to struggle, seems related to #1893)
* Some (most?) players will not return results containing the different glyph in the tag if you type a simple punctuation character in the search field. ie typing "hyphen-minus" on the keyboard will not match "hyphen". (I use quodlibet and it treats these as different.)
* I use Linux and the command line renders the UTF-8 characters fine, but I have the same "gotcha" when I go to type the glyphs.
* Musicbrainz doesn't seem to be entirely consistent in how it applies these. For example, I have some tags "El-P" and some tags "El‐P" (UTF-8 hyphen vs. ASCII hyphen-minus).
I know that I can fix this for files by enabling "asciify", and it looks like this was dealt with for the Lyrics plugin in #270. However as well as Latin-character albums I also have a bunch with names in non-Latin script, so I actually want Unicode for things which I can't effectively represent in ASCII.
I guess my dream feature would be a "sanitise punctuation" option where these almost-the-same-as-an-ASCII-character punctuation glyphs get swapped for their ASCII versions in both tags and filenames, but anything else gets left as UTF-8.
I understand that this is a lot more to do with the design of Unicode than the design of beets (and that some people actually care about the distinction between hyphen-minus and hyphen, I just don't care in this case!)
I'd be happy to look into writing a patch for a feature like the above, if that's potentially acceptable. The approach discussed in #270 for lyrics (ie find-replace) seems applicable.
### Setup
* OS: Linux
* Python version: 3.6.1
* beets version: 1.4.3
* Turning off plugins made problem go away (yes/no):
Contributor guide
Research direction
No source files, tests, or entry points are named in the issue, so begin by locating the code that writes imported metadata and filenames. Done means punctuation with ASCII equivalents is normalized in both tags and filenames while characters without an effective ASCII representation remain unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100