flavor_text_entries contain newline and other characters within the string
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 1.2k
- Avg merge
- 2d 5h
- Merged PRs (30d)
- 21
Description
Summary
- The source data for
flavor_textuses characters like\nand\u000cwhich make it difficult to read normally - PokéAPI contains the same characters which might not be desirable
- Suggest to parse the raw text into a readable format for the benefit of most common use-cases
- If not, consider adding this info into the docs so users might have a chance to know about it and handle the data appropriately
Problem
Some flavor_text_entries have issues with additional unexpected characters.
Example from Wobbuffet's entry:
"flavor_text_entries" : [
{ "flavor_text" : "It hates light and\nshock. If attack\ned, it inflates\u000cits body to pump\nup its counter\nstrike." }
...
{ "flavor_text" : "To keep its pitch-\nblack tail hidden,\nit lives quietly\u000cin the darkness.\nIt is never first\nto attack." }
...
{ "flavor_text" : "In order to con\nceal its black\ntail, it lives in\u000ca dark cave and\nonly moves about\nat night." }
...
]
Seems like it is also not limited to PokemonSpecies, nor to language. See the move Seismic Toss:
"Inflicts damage identical\nto the user’s level."
"L’ennemi est projeté grâce au pouvoir de\nla gravité. Inflige des dégâts équivalents\nau niveau du lanceur."
"いんりょくを つかい なげとばす。\nじぶんの レベルと おなじ ダメージを\nあいてに あたえる。"
Details of the problem
These characters seem to originate from the CSV file data from veekun/pokedex. A comment on this closed issue about "parsing pokemon_species_flavor_text.csv" confirms that the presence of these characters within the files is the intended behaviour, to reflect the original format of the text within the game files, which contain "explicit line breaks, hyphenation, and page breaks".
There is also a suggested method to replace these characters to make it more readable, although it is a little more complicated than simply "replace \n with whitespace/empty string".
Impact and Suggestions
Above is an example of someone's application using the unprocessed flavor_text directly. which has an unfortunate result of making it look somewhat unrefined. It's an avoidable outcome which the PokéAPI team can perhaps help mitigate.
The team may want to implement parsing of the flavor_text for the API, since I would expect that most users of the API would simply take the flavor_text as it is, assuming that it is formatted correctly.
Alternatively, if it is neither feasible nor desirable to do this parsing on the API side, then perhaps it should be stated within the documentation that the flavor_text are in an unprocessed form which will require parsing on the user's side. As far as I can tell, this information is currently not being reflected in the docs. If it was, then at least some users would take note and be able to process the data on their end.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked source CSV data and compare the flavor_text values returned by the Pokémon species and move API endpoints. Decide whether the project should normalize the characters in API output or document their raw format; done means the chosen behavior is consistently applied or clearly explained to users.
Written by the indexing model from the issue text.
Assessment
- Domain
- api, data, documentation
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100