Bayesian Framework for Classification of Sensor Data

back to notes

Bayesian Analysis

• Given a prior distribution,
• collect data (samples) to get an observed distribution,
• then calculate the observed distribution's likelihood as a function of parameter values,
• multiply this likelihood function by the prior distribution,
• normalize to obtain a unit probability over all possible values
• This is called the posterior distribution.

Bayes Theorem

The aim is to find the probability that a certain label is the right classification for the current input vector x. This is called the posterior probability:

from Bayes' Theorem:

• is the probability that (x₁,...,x_d) is being measured by the sensors, when we know that l_i is the current label.

• is the probability that the current label is l_i

• is the probability that the values x₁,...,x_d are being measured by the sensors

Maximum A Posteriori (MAP) hypothesis

We can leave out since it is just a constant, and would not affect the maximum l_i , and we could even leave the out if we assume that all labels are equally probable: with n the number of labels. This leads to.. 

Maximum Likelihood (ML) hypothesis

The term is often called the likelihood of (x₁,...,x_d) given the class l_i , hence the name maximum likelihood.

Naive Bayes (NB) classifier

The first term of the MAP hypothesis could be simplified if the assumption is made that all sensors are conditionally independent, if the correct label is given: . In this case the Naive Bayes classifier is thus:

This assumption is usually not very workable when dealing with multiple sensors, though. Conditionally independent means that we can write: .