fslaborg / fslaborg/FSharp.Stats

[Feature Request] `Frequency` merge operations

Open
#263 3 comments 0 reactions 2 assignees Claimed by @bvenn View on GitHub
enhancement
Dominant language
F#
Stars
227
Forks
58
Avg merge
55m
Merged PRs (30d)
1

Description

## Merge operations for Maps

Data can be sorted into bins of predefined width using the `Frequency` or `EmpiricalDistribution` module. If two datasets are binned and should be merged afterwards, several merging strategies are possible. A simple merge of `freqA` and `freqB` is straightforward with keys that are present in `freqA` and `freqB` are replaced with the values of `freqB`.

```fsharp
let a =
[("k1",1);("k2",3)]
|> Map.ofList

let b =
[("k2",1);("k3",4)]
|> Map.ofList

merge a b
```
results in the following combination with `("k2",3)` from `a` being replaced by `("k2",1)` from `b`:

```fsharp
val it: Map = map [("k1", 1); ("k2", 1); ("k3", 4)]
```

### Generic formulation of merge operations

I'm in the process of adding a generic function that gets an additional function that handles key duplicates. E.g.:

```fsharp
add a b
```
resulting in the combination of a and b with `("k2",3)` from `a` being added to `("k2",1)` from `b`:

```fsharp
val it: Map = map [("k1", 1); ("k2", 4); ("k3", 4)]
```

While this is trivial, I'm not sure how to handle a subtraction. Should the result from `subtract a b` result in:
- a) `val it: Map = map [("k1", 1); ("k2", 2); ("k3", 4)]`
- counts from `a` are subtracted by the corresponding values from `b` if keys are present in both maps
- here the values of `a` that are not present in `b` are untouched
- b) `val it: Map = map [("k1", 1); ("k2", 2); ("k3", -4)]`
- counts from `a` are subtracted by the values from `b`, even for keys that are not present in `a`

The latter option (b) makes no sense to me since frequency counts should not be negative, but I cannot think of applications in which the result of (a) makes any sense. Maybe the subtract function is not the best to start with because in this [post](https://stackoverflow.com/questions/38987/how-do-i-merge-two-dictionaries-in-a-single-expression-in-python) they implemented (a) with addition and multiplication examples. Especially for the addition, a and b would give the correct result and I think it is intuitive to just apply the function to values of keys that are present in both maps.

@HarryMcCarney, do you know use cases that use `subtract`? Do you have any thoughts about this? I would suggest to add version (a) to `Frequency` as well as `EmpiricalDistribution`

Additional remark: When applied to continuous data bandwidths must be equal, to not merge counts from overlapping bins!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.