I need to determine the KL-divergence between two Gaussians. I am comparing my results to these, but I can't reproduce their result. My result is obviously wrong, because the KL is not 0 for KL(p, p).
I wonder where I am doing a mistake and ask if anyone can spot it.
Let and . From Bishop's PRML I know that
where integration is done over all real line, and that
so I restrict myself to , which I can write out as
which can be separated into
Taking the log I get
where I separate the sums and get out of the integral.
Letting denote the expectation operator under , I can rewrite this as
We know that . Thus
which I can put as
Putting everything together, I get to
Can anyone spot my error?
Thanks to mpiktas for clearing things up. The correct answer is: