CentreForDigitalHumanities / CentreForDigitalHumanities/Textcavator

Duplicate code for wide format tables

Open
#1,067 0 comments 0 reactions 0 assignees View on GitHub
code quality
Dominant language
Python
Stars
12
Forks
3
Avg merge
3d 3m
Merged PRs (30d)
9

Description

Minor issue.

Our visualisation CSVs are long format by default, and the frontend and backend both implement functionality convert them to wide format. The frontend does it for tables shown in the interface, the backend does it for downloads. This should provide the same output, but the underlying code is completely separate.

This happened because the frontend version was introduced first, but at this point, it would make more sense to run all wide format conversion through the backend.

Note: The frontend uses a typescript implementation based on lodash. The backend uses pandas, which means the script mostly consists of workarounds for pandas idiosyncracies. I would prefer to take the frontend implementation and "translate" it to python. If the backend conversion was based on lists of dicts (rather than pandas objects), it could run on JSON tables from the frontend without issue.

Contributor guide

No contributing guide indexed for this repository

Research direction

Compare the frontend TypeScript implementation based on lodash with the backend Python conversion based on pandas. Start by locating both wide-format conversion paths and verify their outputs on the same table data. Done means the backend provides the shared conversion used for downloads and frontend tables, with matching output.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python, typescript
Domain
data, full-stack
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.