More consistency in ETL functions regarding `data["attribute"]` vs `data.get("attribute")`
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 3
- Forks
- 4
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 114
Description
Description/Context
Some ETL transform functions use the format source_data["attribute"] for some/all attributes- this is likely because the attribute is always supposed to be present in the source data, and the function should fail if it isn't, to indicate that something is wrong with the source data.
Other functions use the format source_data.get("attribute") for some/all attributes - likely because the attribute might not always be present in the source data, and this is considered okay for the particular source.
It might be good to double-check on these and make sure they are still used appropriately per data source and per attribute, and make them more consistent where applicable.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Locate the ETL transform functions and inventory uses of source_data["attribute"] and source_data.get("attribute") by data source and attribute. Read the surrounding source schemas and existing tests, if present, before deciding which accesses should fail when missing. Done means the access style matches each attribute's intended presence requirements and relevant behavior remains covered.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100