Have you ever asked yourself how does mail spam filtering actually work? It is related with Bayesian theorem. Bayesian theorem is related to conditional probability, probability in relation with previous events. Many computer events are conditional, meaning Bayesian theorem can be applied to them. So what is Bayesian theorem?
Short introduction about probability theory
Classical theory of probability
If you roll a dice 10 times and the outcome is 6 all the time in sequence, what is the probability of rolling number 6 the eleventh time ? Well it does not depend what were the previous outcomes. It will be always ⅙. Rolling a dice is not a conditional probability. If you roll dice 20 times and there was no outcome of number 6 then some people believe that 21th time there is bigger chance to get number 6. That is not true. Again the probability is ⅙. This is cognitive bias of thinking about classical theory of probability.
Let’s take a look at another example. One factory produces cubes with side-length between 0 and 1 meters. What is the probability that a randomly chosen cube has side-length between 0 and ½ meters? If the process of production that is uniformly distributed over side-length, then the probability is 0.5, which is correct.
Let’s look at the same problem from another point of view. A factory produces cubes with face-area between 0 and 1 square meters; what is the probability that a randomly chosen cube has face-area between 0 and ¼. In this case the probability is 0.25.
Wait a minute. Cube with side of 0.5 meters has face area of ¼ (0.25 square meters). For the same cubes we get different results, once 0.5 and then 0.25. Well the actual result is 0.5. This is called Bertrand’s Paradox or cube paradox. http://en.wikipedia.org/wiki/Talk%3ABertrand’s_paradox_(probability)
The above sample tells us that we can easily make a mistake with the classical probability theory in real life. The classical probability is good for gambling games, but it can be very tricky for real life situations. Also real life situations are conditional. Some event B can be dependent from event A. In that case we need Bayesian. But even conditional probability is wrongly understood by many.
I will not go into mathematical details about Bayesian theory. There are plenty of resources about the Bayesian Theory (http://en.wikipedia.org/wiki/Bayes’_theorem) and Conditional probability (http://en.wikipedia.org/wiki/Conditional_probability)
Conditional probability is very important even in real life, but not well understood by people when discussing about it. Conditional probability has a very important role in deductive reasoning and is often used in the courts, computer science and many other fields. I will explain how conditional logic is misunderstood with an example of a medical test.
When they tell you that the test has probability of 99%, does that mean that chances that you are really sick is 99%. Lets see.
Let’s say you are randomly chosen to make AIDS test.
Let’s say that AIDS test has probability of 99% (1% error) But If you like to test randomly chosen persons, the test can be applied to both ill and healthy person (we do not know if the person has or does not have aids.). Also it is given that only 0.4% of the population have AIDS.
I will solve the problem with another approach (more visualized, but the same result can be calculated with empirical formula):
P(HIV|+) is a probability of having the virus after positive test. In this case for test with accuracy of 99% and given 0.4% infected population, the probability of of having AIDS is 28% after positive test (not 99%!) The probability is calculated as a number of true diagnosed infected persons divided by total positive diagnosed people (including healthy people diagnosed as infected). This result is for given 0.4% of the population have AIDS. This is the condition for which the Bayesian is calculated.
Note that if the test was 100% accurate, then also Bayesian probability would be also 100%.
There are plenty of other examples how Bayesian is used in the real world.
Bayesian in the computer world
Because many events in the computer world are conditional, to find the real probability a Bayesian principle must be applied. For your preview I will mention where Bayesian is applied.
Bayesian is widely used in Spam filtering, artificial intelligence, pattern recognition/face detection. It is also used in statistical classification and traffic analysis.
You should definitely take a look at Bayesian probability. It might suit somewhere in your programming problems.