queryverse / queryverse/ReadStat.jl
Reading `.dta` with value labels
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 79
- Forks
- 18
- Avg merge
- 22h 31m
- Merged PRs (30d)
- 3
Description
As you know, Stata basically stores value-labeled data as a vector of integers or doubles, not necessarily an ordered sequence starting at 1, and a Dict going from Int => String.
Accessing the string values, which we generally care the most about, is hard with ReadStat. You have to
- Use ReadStat not StatFiles to access the internal fields of the Stata File
- Construct the
DataFamefrom the data and header fields
3 . Use thevalue_label_dictfield to perform the replacement - Use
geton the DataValue elements of the array
This is not the most user friendly thing.
There isn't a great solution for this in Julia as we dont have a CategoricalArray equivalent where the base dict maps arbitrary types to strings. So converting to categorical array will drop the underlying integers, which are useful to keep due to inter-operability.
haven in R recently made a change with how this is handled with the <dbl+lbl> vector type. Though working with it is a bit of a pain, see here.
I can email a data-set to someone with an MWE for more information.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing how ReadStat exposes Stata value-label fields and how StatFiles currently constructs the DataFrame, including value_label_dict and DataValue access. Compare the issue's described workflow with haven's <dbl+lbl> approach; done should provide a user-friendly way to access labels while preserving the underlying integer or double values.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100