fslaborg / fslaborg/FSharp.Stats
[Feature Request] `Frequency` merge operations
- Dominant language
- F#
- Stars
- 227
- Forks
- 58
- Avg merge
- 55m
- Merged PRs (30d)
- 1
Description
## Merge operations for Maps
Data can be sorted into bins of predefined width using the `Frequency` or `EmpiricalDistribution` module. If two datasets are binned and should be merged afterwards, several merging strategies are possible. A simple merge of `freqA` and `freqB` is straightforward with keys that are present in `freqA` and `freqB` are replaced with the values of `freqB`.
```fsharp
let a =
[("k1",1);("k2",3)]
|> Map.ofList
let b =
[("k2",1);("k3",4)]
|> Map.ofList
merge a b
```
results in the following combination with `("k2",3)` from `a` being replaced by `("k2",1)` from `b`:
```fsharp
val it: Map = map [("k1", 1); ("k2", 1); ("k3", 4)]
```
### Generic formulation of merge operations
I'm in the process of adding a generic function that gets an additional function that handles key duplicates. E.g.:
```fsharp
add a b
```
resulting in the combination of a and b with `("k2",3)` from `a` being added to `("k2",1)` from `b`:
```fsharp
val it: Map = map [("k1", 1); ("k2", 4); ("k3", 4)]
```
While this is trivial, I'm not sure how to handle a subtraction. Should the result from `subtract a b` result in:
- a) `val it: Map = map [("k1", 1); ("k2", 2); ("k3", 4)]`
- counts from `a` are subtracted by the corresponding values from `b` if keys are present in both maps
- here the values of `a` that are not present in `b` are untouched
- b) `val it: Map = map [("k1", 1); ("k2", 2); ("k3", -4)]`
- counts from `a` are subtracted by the values from `b`, even for keys that are not present in `a`
The latter option (b) makes no sense to me since frequency counts should not be negative, but I cannot think of applications in which the result of (a) makes any sense. Maybe the subtract function is not the best to start with because in this [post](https://stackoverflow.com/questions/38987/how-do-i-merge-two-dictionaries-in-a-single-expression-in-python) they implemented (a) with addition and multiplication examples. Especially for the addition, a and b would give the correct result and I think it is intuitive to just apply the function to values of keys that are present in both maps.
@HarryMcCarney, do you know use cases that use `subtract`? Do you have any thoughts about this? I would suggest to add version (a) to `Frequency` as well as `EmpiricalDistribution`
Additional remark: When applied to continuous data bandwidths must be equal, to not merge counts from overlapping bins!
Contributor guide
Assessment
This issue has not been assessed yet.