ENH tools, distance measures for distributions, find closest approximation

The issue does not name files, tests, or an entry point. Start by reviewing the existing goodness-of-fit tools, then compare the proposed CDF distances, quantile matching, and KL-based approaches. The scope, API, supported methods, and completion criteria need to be agreed before implementation can begin.

Some distributions have no closed form (or special) versions and are not tractable. For some use cases we can use approximation that are easier to work with.

example: moment matching for p-values in hypothesis tests. recent examples that I used was shifted and scaled chisquare approximation to distribution of weighted sum of chisquare random variables, (e.g. LR tests under misspecification or incomplete specification.)

There is also a literature on finding approximations for quantile function, ppf, to make simulating random numbers easier.

In the literature for GLM link functions for binomial I saw it mentioned that t-distribution with around 6 or 7 degrees of freedom is close to logistic distribution.

Tail approximation: extreme value theory looks at distribution of the tail and which distribution provides good approximation to the tail behavior, e.g. pareto tails.
One question is whether a distribution has power tails, exponential tails or some other tails. This is important if we are not just interested in approximating the main part of a distribution.

this article includes a survey of methods to stich a distribution for the tail together with a distribution for the bulk
Scarrott, Carl, and Anna MacDonald. "A review of extreme value threshold estimation and uncertainty quantification." REVSTAT–Statistical Journal 10, no. 1 (2012): 33-60.

Tools:

I'd like something similar to gof test, but for distributions, e.g. match two cdfs, where one is the reference and the other is parameterized to match as closely as possible.
In this case we need to define a distance measure similar to KS test, "sup", integral versions like Cramer, von Mises or Anderson, Darling, or matching just a few quantiles or values of cdf. Then we can minimize the distance to find best matching parameters.

Other alternative, Kullback-Leibler distance. In this case we could also use MLE which minimizes KL distance in misspecified models.

related: