Independence and conditional independence are one of the most basic concepts in probabilities. Two random variables are independent, as the term suggested, if the outcome of one variable should not affect the outcome of another. Mathematically, two variables and are independent if
for any outcome for variable and outcome for variable . And we often indicate this independece by .
Let’s inspect this definition more carefully, given and by Bayes’ rule, the probability of is
. Indeed the probability does not depend on the outcome of and so and are independent.
Similarly, when we say and are conditionally independent given , it means that knowing , and become independent. Note that and do not need to be independent to start with. Many students have the misconceptions that independence implies conditional independence, or vice versa. The fact is that they are completely unrelated concepts. We can easily find examples that variables are independent but not conditional independent and vice versa. And also examples that variables are both independent and conditional independent. And of course cases when neither independence is satisfied.
Mathematically, we denote the independence by , and we have
for any , and .
Note that the definition above implies that . Hence, indeed if we are given , the probability of does not dependent on the outcome of . So it depicts well the conditional independence that we anticipated.