Bayes’ theorem can be expressed as following
$$ P(H|E) = \frac{P(E|H) \times P(H)}{P(E)} $$
But what exactly does this formula mean?
Let's consider a scenario: Imagine there's a rare disease affecting 1 in every 10,000 people in a population of 1 million. You undergo a test for this disease, and the test is reported as positive by your doctor, which might be alarming. However, before jumping to conclusions, let’s review the the critical numbers:
Great, now let's try to determine the probability of actually having the disease given a positive test result.
Step 1:
We start with a hypothesis, $H$, that you have the disease.
We want to determine your initial belief of the probability of the hypothesis being true, i.e. $P(H)$. This is known as the prior probability. And we are given the information that 1 in 10,000 people are affected by the disease → $P(H) = 0.0001$
Step 2:
We are also given new information, also known as the evidence, $E$, that you are reported positive.
We want to know that out of the probability of the evidence, how much corresponds to the hypothesis. The probability of the evidence occuring is known as the marginal probability, $P(E)$.
We may think that $P(E) = 0.99$ because the test accurately diagnoses 99 out of 100 sick individuals. But we would have failed to consider that it is possible you were not sick but reported positive. So we have two scenarios: