easystats / easystats/datawizard

Rethinking `degroup()` for cross-classified data

Open
#637 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Consistency :green_apple: :apple: Discussion :parrot:
Dominant language
R
Stars
238
Forks
18
Avg merge
1d 15h
Merged PRs (30d)
4

Description

Follow-up up @jmgirard's #520

I think the current implementation of cross-classified disaggregation is missing a desiderata.

First let's note some desiderata that we do have:

  1. The "between" variable are simply the separate group means.
Make some crossed data
mu <- 100
ul <- setNames(c(-1, -3, 0, 4), nm = letters[1:4])
uL <- setNames(c(10, 30, 0, -40), nm = LETTERS[1:4])
um <- setNames(c(100, 150, -250), nm = month.abb[1:3])

dat <- expand.grid(l = letters[1:4], L = LETTERS[1:4], m = month.abb[1:3])

set.seed(111)
e <- rnorm(nrow(dat)-1) |> round(2)
e <- append(e, -sum(e))

dat$y <- mu + ul[dat$l] + uL[dat$L] + um[dat$m] + e
dat$z <- mu + ul[dat$l] + uL[dat$L] + um[dat$m] + 10*e
dat_dem <- datawizard::demean(dat, by = c("l", "L", "m"), select = c("y","z"))

all.equal(c(dat_dem$y_l_between), ave(dat$y, dat$l))
#> TRUE
all.equal(c(dat_dem$y_L_between), ave(dat$y, dat$L))
#> TRUE
all.equal(c(dat_dem$y_m_between), ave(dat$y, dat$m))
#> TRUE
  1. The sum of an observation's "between"/"within" variables is equal to the original observation
all.equal(rowSums(dat_dem[grepl("^y_", colnames(dat_dem))]), dat$y)
#> TRUE

What we don't have is that -- unlike with a single grouping variable or with nested designs -- the "within" variable is mean centered:

mean(dat_dem$y_within)
#> -200

This is equal to $(-\bar{Y})\times (\text{number of grouping vars} - 1)$

-mean(dat$y) * (3-1)
#> -200

I think this is something we want, for consistency (typically "within" is considered to be automatically double-centered), however with a crossed design this cannot be achieved without compromising on desiderata 1 or 2.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the supplied R example with datawizard::demean() and inspect the current degroup() behavior for cross-classified data. No file or test is named; done requires an agreed behavior for mean-centering the within component while addressing the stated tradeoff with the between values and reconstruction invariant.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.