Test case(s) for fastmean.c based on mutation testing results; Base R's mean vs data.table's fast mean

Open
#6,176 15 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
c, r
Domain
testing-qa

Research direction

Start by reading src/fastmean.c around the s += t/n expression and the Cfastmean entry point referenced in R/data.table.R. Compare existing coverage in test.data.table() and the mutation-related cases from #6114 and #6115, then run the relevant R tests with inputs that distinguish division from multiplication. Done means a test fails for the mutant and passes for the original implementation.

Written by the indexing model from the issue text.

Description

tests

I'm trying to create a test that will fail for a mutant of fastmean.c that involves an arithmetic operator change as discovered/mentioned in #6114 but I'm stuck on creating a viable test case for it as everything I've tried so far returns the same result for both cases: (or for both with and without this change)

s += t/n; // original
s += t*n; // mutant

Also, I'm wondering if my procedure in going about this so far is correct or not - Initially, I went about recompiling that C code via R CMD SHLIB fastmean.c (gcc with a path spec for the R header works too, but this was faster) after making changes to that file to get the shared object and then loaded that .so into R via dyn.load() with a wrapper to call the C function via .Call, given that there isn't an exported object in the data.table namespace for fast mean directly.

I then found a better/faster way to do it after doing a search to see it's usage in R code:

grep -r "fastmean" R/ tail -n 1
R//data.table.R:    return(call(".External",quote(Cfastmean),expr[[2L]], expr[[3L]]))  # faster than .Call

I switched to using .External to call that C routine via the "Cfastmean" symbol and that's how I'm currently running fastmean.c in my tests with REALSXP type inputs for that execution path, but I haven't found success in being able to create a suitable test for it that would fail for the mutant and pass for the original code. (I'm trying to come up with something similar to the other test case introduced based on mutation testing results - #6115, where the relation operator replacement triggers the condition with empty keys, leading to incorrect behaviour by executing code that assumes there is a single key)

Related testing - I was able to create test cases for which the output of base::mean did not match with the Cfastmean computed result on some checks though, like the one below:

library(data.table)

meanComparison <- function(x, ...) 
{
    baseR <- mean(x, ...)
    fastmean <- .External("Cfastmean", x, ...)
    cat("Results as computed by:\nBase R's mean:", baseR, "\ndata.table's fast mean:", fastmean, "\n")
    fifelse(identical(baseR, fastmean), "Passed", "Failed")
}

maxDoubleValue <- .Machine$double.xmax
testInputs <- c(maxDoubleValue, maxDoubleValue, -maxDoubleValue, -maxDoubleValue)
cat(meanComparison(testInputs), "the mean comparison test with maximum-value doubles as inputs.\n")
Results as computed by:
Base R's mean: 0 
data.table's fast mean: Inf 
Failed the mean comparison test with maximum-value doubles as inputs.

A corner case, but thought to share this since among the starting comments in fastmean.c I saw "We explicitly test that fastmean returns the same result as base::mean in test.data.table()." (which like this case, I assume might not be necessarily true for certain inputs or ditto given some possible inconsistency in handling such values wrt base R's mean implementation?)

Dominant language
R
Stars
3.9k
Forks
1.1k
Avg merge
14h 4m
Merged PRs (30d)
4

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Rdatatable/data.table

All issues in Rdatatable/data.table

Similar issues

More R issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.