Now suppose we pick a box at random, and it turns out to be a blue box. Then, choose the probability of an apple is the apple in the blue box scores, namely 3/43/43/4, so p (F = a = B ∣ B) = 3/4 p (F = = B a | B) = 3/4 p (F = a = B ∣ B) = 3/4. In fact, we can write down all four conditional probabilities for the fruit type, given the box we picked


p ( F = a ∣ B = r ) = 1 / 4 (1.16) P (F = = r a | B) = 1/4 \ tag} {1.16

p ( F = o ∣ B = r ) = 3 / 4 (1.17) P = o | B = r (F) = 3/4 \ tag} {1.17

p ( F = a ∣ B = b ) = 3 / 4 (1.18) P (F = = B a | B) = 3/4 \ tag} {1.18

p ( F = o ∣ B = b ) = 1 / 4 (1.19) P = o | B = B (F) = 1/4 \ tag} {1.19

Notice again, these probabilities are normalized, so


p ( F = a ∣ B = r ) + p ( F = o ∣ B = r ) = 1 (1.20) P = a | B = r (F) + p (F = o | B = r) = 1 \ tag} {1.20

The same


p ( F = a ∣ B = b ) + p ( F = o ∣ B = b ) = 1 (1.21) P (a | B F = = B) + p (F = o | B = B) = 1 \ tag} {1.21

We can now use the sum product rule of probability to evaluate the overall probability of choosing an apple


p ( F = a ) = p ( F = a ∣ B = r ) p ( B = r ) + p ( F = a ∣ B = b ) p ( B = b ) = 1 4 x 4 10 + 3 4 x 6 10 = 11 20 (1.22) p(F=a)=p(F=a|B=r)p(B=r)+p(F=a|B=b)p(B=b)=\frac{1}{4}\times\frac{4}{10}+\frac{3}{4}\times\frac{6}{10}=\frac{11}{20}\tag{1 22}.

Thus, the use of sum rule, p (F = o) = 1 – p (F = o) = 11/20 = 9/20 9/20-11/20 = 1 p (F = o) = 1-11/20 = 9/20.

Instead, suppose we’re told we’ve chosen a piece of fruit, and it’s an orange, and we want to know which box it came from. This requires us to evaluate the probability distribution on the box conditioned on the identity of the fruit, and the probability in (1.16) (1.19) gives the probability distribution on the fruit conditioned on the identity of the box. Using Bayes’ theorem, we can solve the inverse problem of conditional probability


p ( B = r ∣ F = o ) = p ( F = o ∣ B = r ) p ( B = r ) p ( F = o ) = 3 4 x 4 10 x 20 9 = 2 3 (1.23) P (B = r | = o F = \ frac {p (F = o | B = r) p (B = r)} {p (F = o)} = \ frac {3} {4} \ times \ frac {4} {10} \ times \ frac {20} {9} = \ frac {2} {3} \ tag} {1.23

According to the sum rule, p (B ∣ f = B = o) = 1 – two-thirds of 1/3 = p (a | B = B = o f = 1 – two-thirds of 1/3 = p (B ∣ f = B = o) = 1-2/3 = 1/3.

We can make the following important interpretations of Bayes’ theorem. If we are asked which box we chose before being told the identity of the fruit we chose, the most complete information we can get is the probability p(B)p(B)p(B). We call it a prior probability, because it’s the probability available before we look at the properties of the fruit. Once we were told that the fruit is orange, we can use bayes’ theorem to calculate the probability p (B ∣ F) p (B | F) p (B ∣ F), we called it a posteriori probability, because it was after we observed FFF probability. Note that in this example, the prior probability of choosing the red box is 4/104/104/10, so we are more likely to choose the blue box over the red box. However, once we look at the chosen fruit when it is an orange, we find that the posterior probability of the red box is now 2/32/32/3, so the box we are now more likely to choose is actually red. This is intuitive because the proportion of oranges in the red box is much higher than that in the blue box, so the observed fruit provides important evidence in favor of oranges in the red box. In fact, the evidence was strong enough to exceed previous evidence to make the red box more likely than the blue one.

Finally, we note that if the joint distribution of the two variables is decomposed into a product of edges such that P (X,Y)= P (X)p(Y)p(X,Y)=p(X)p(Y)p(X,Y)= P (X)p(Y)p(X,Y)= P (X)p(Y) then XXX and YYY are said to be independent. From product rules, we see p (Y ∣ X) = p (Y), p (Y | X) = p (Y), p (Y ∣ X) = p (Y), so the YYY of the conditional distribution of a given XXX really has nothing to do with the value of the XXX. In our example, fruit box, for example, if each box containing the same proportion of apples and oranges, then p (F ∣ B) = p (F) p (F | B) = p (F) p (F ∣ B) = p (F), so the selection (for example) the probability of apple has nothing to do with the choice which box.