On density estimation parameters interpolation and bias

[jiayily]

What do interpolation and bias mean in density estimation? Can you please give a detailed comment in the code?

[charliematteo]

Hi @jiayily,

Thank you for your request. The reference to look at is this one: https://arxiv.org/abs/2102.06167

We will comment the relevant parts soon, thanks for pointing it out. For the moment, I try to answer briefly.

All the density estimators in DADApy, such as PAk or kstarNN, produce pointwise estimates of the logdensity on the N training points. If you want to know the density at a new test point which is not included in those N, you need to use the interpolate function, which models the density off-sample.

Then the bias. Taking the language from statistical mechanics canonical ensable, we can call the negative log-probability-density the potential energy U(x). Any constant factor that you add to U(x) is absorbed by the normalisation of p. So p(x) ∝exp[-U(x)] or equivalently U(x) = - log[p(x)] + const. Now suppose p is hard to sample, then we can add a bias potential B(x) to U(x) in order to make their sum V=U+B smoother than U itself, i.e. to make p_bias(x) ∝exp[-V(x)] easier to sample w.r.t. p. However, when you estimate the density from your sample you may still be interested in p and not in p_bias. So you have to "reweight" your estimates keeping track of the bias potential applied when sampling. If you specify a bias B(x_i) you applied when sampling any given point x_i, the algorithm in DADApy will correctly reweight the contribution and estimate log p, assumed that B(x) depends only on x.

I will leave the issue open until comments are added.

Best

[jiayily]

@charliematteo Great! Thanks for your detailed explanation. It helps a lot.