Here I will derive Kalman filter from belief propagation (BP). While I won’t derive exactly the closed-form expression as Kalman filter, one can verify numerically that the BP procedure below is the same as the standard Kalman filter. For derivation to the exact same expression, please see here.
Kalman filter is a special case of Gaussian BP. It can be easily understood as three different propagation steps as below.
Scenario 1: Passing message from to for
Say , . And as
Scenario 2: Passing message from back to for
This is a bit trickier but still quite simple. Let the covariance of be . And from the perspective of ,
Scenario 3: Combining two independent Gaussian messages
Say one message votes for and precision (precision is the inverse of the covariance matrix). And say another message votes for and precision .
BP for Kalman update
Let’s utilize the above to derive the Kalman update rules. We will use almost exactly the same notation as the wiki here. We only replace by and by to reduce cluttering slightly.
- As shown in the figure above, given and , we have as in (1)
as it is almost identical to Scenario 1 besides a trivial constant shift of .
- Then step (2) is obvious as, is nothing but added by a noise of covariance .
- Now, step (3) is just a direct application of Scenario 2.
Finally, we should combine the estimate from (2) and (3) as independent information using the result in Scenario 3. This gives us the final a posterior estimate of as
Although it is not easy to show immediately, one can verify numerically that the and the mean above are indeed and in the original formulation, respectively.