stats: std population or sample calculation on 1 value
- Dominant language
- JavaScript
- Stars
- 18
- Forks
- 10
- PR merge metrics
- No merged PRs in 30d
Description
Calling `th.stats.std(ttk, cols)` may result in aggregation of a single row. In thicket, we are calling `.agg(np.std)`, which calculates the standard deviation with a degrees of freedom (`ddof`) of 1. In other words, it divides by `n-1` (where n is the number of elements). This is statistically appropriate for estimating the population standard deviation from a sample. However, with only one element, the calculation becomes 0/0, resulting in a NaN value.
One alternative is to set `ddof=0`, calculating the standard deviation with ddof=0, dividing by n instead of n-1, resulting in a standard deviation of 0 for a single element:
```
import pandas as pd
import numpy as np
df = pd.DataFrame({'A': [1]})
# Calculate standard deviation with ddof=0
result = df.agg(lambda x: np.std(x, ddof=0))
print(result)
```
For standard deviation, it may be appropriate to have an option to toggle between population and sample calculation.
Contributor guide
Research direction
Start at the th.stats.std(ttk, cols) entry point and trace the .agg(np.std) call described in the issue. Confirm the current single-row result, then determine how the population-versus-sample choice should be represented; the work is done when the selected behavior is defined and the one-value aggregation no longer produces an unintended result.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, pandas, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100