CentreForDigitalHumanities / CentreForDigitalHumanities/Textcavator
Duplicate code for wide format tables
- Dominant language
- Python
- Stars
- 12
- Forks
- 3
- Avg merge
- 3d 3m
- Merged PRs (30d)
- 9
Description
Minor issue.
Our visualisation CSVs are long format by default, and the frontend and backend both implement functionality convert them to wide format. The frontend does it for tables shown in the interface, the backend does it for downloads. This should provide the same output, but the underlying code is completely separate.
This happened because the frontend version was introduced first, but at this point, it would make more sense to run all wide format conversion through the backend.
Note: The frontend uses a typescript implementation based on lodash. The backend uses pandas, which means the script mostly consists of workarounds for pandas idiosyncracies. I would prefer to take the frontend implementation and "translate" it to python. If the backend conversion was based on lists of dicts (rather than pandas objects), it could run on JSON tables from the frontend without issue.
Contributor guide
No contributing guide indexed for this repository
Research direction
Compare the frontend TypeScript implementation based on lodash with the backend Python conversion based on pandas. Start by locating both wide-format conversion paths and verify their outputs on the same table data. Done means the backend provides the shared conversion used for downloads and frontend tables, with matching output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python, typescript
- Domain
- data, full-stack
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100