Bayesian Statistics

- Overview

Bayesian statistics is a statistical method that uses probability distributions to describe the state of knowledge about unknown quantities. It is based on the Bayesian interpretation of probability, where probability expresses a degree of belief in an event. 

Bayesian statistics is used to calculate the probability that an alternative is superior. It is widely used across the social, biological, medical, and physical sciences. For example, Nate Silver used Bayesian statistics to correctly predict the 2008 US presidential election results. 

Bayesian statistics uses Bayes' theorem to update the probability for a hypothesis as more evidence or information becomes available. It uses prior knowledge, in the form of a prior distribution, to estimate posterior probabilities. 

Bayesian statistics is unique in that all observed and unobserved parameters in a statistical model are given a joint probability distribution. This distribution is termed the prior and data distributions.

Please refer to the following for more information:

• Wikipedia: Bayesian Statistics
• University of Washington: Bayes' Theorem

- The Formula for Bayes' Theorem

Bayes' theorem is stated mathematically as the following equation:

where A and B are events and P(B)≠0.

Bayes' theorem can be expressed as P (A|B) = P (B|A) * P (A) / P (B). P (A|B) is the probability that A will occur given that B occurs. P (B|A) is the probability that B will occurs given that A occurs. P(A) and P(B) are the probabilities of observing A and B respectively without any given conditions; they are known as the prior probability and marginal probability.

Bayes' theorem is a mathematical formula used to determine the conditional probability of events. It's also known as Bayes' rule and is named after English statistician Thomas Bayes, who discovered the formula in 1763.

- Characteristics of Bayesian Statistics

Bayesian statistics is a statistical approach that uses probability distributions to describe knowledge about unknown quantities. It also uses Bayes' theorem to update this knowledge based on observed information. 

Here are some characteristics of Bayesian statistics:

• Uses probability distributions: Bayesian statistics uses probability distributions to describe knowledge about unknown quantities.
• Uses Bayes' theorem: Bayesian statistics uses Bayes' theorem for data analysis and parameter estimation.
• Uses joint probability distributions: Bayesian statistics assigns a joint probability distribution to all observed and unobserved parameters in a statistical model.
• Uses conditional probability: Bayesian statistics mostly involves conditional probability, which is the probability of an event A given event B.
• Uses prior and posterior distribution functions: Bayesian statistics is based on prior and posterior distribution functions.
• Uses prior information: Bayesian statistics uses prior information to make models more certain. 

- Activities of Bayesian Statistics 

Bayesian statistics is a mathematical approach to data analysis and parameter estimation that uses Bayes' theorem. It uses probability to express degrees of belief in events. 

Bayesian statistics involves assigning a joint probability distribution to all observed and unobserved parameters in a statistical model. This distribution is known as the prior and data distributions. The posterior distribution then reflects one's updated knowledge, balancing prior knowledge with observed data. 

Bayesian statistics includes a number of activities, including:

• Bayesian inference: Quantifies uncertainty in inferences using probability.
• Statistical modeling: Requires specifying prior distributions for unknown parameters.
• Design of experiments: Uses sequential analysis techniques to include the outcome of earlier experiments in the design of the next experiment.

Bayesian modeling can be used to inform policy decisions by providing a quantitative assessment of a variety of complex risks associated with exposure to pollutants.

[Chicago, USA]

- Bayesian Modeling

Bayesian modeling is a statistical model that incorporates prior knowledge into the model. This prior knowledge is based on information before seeing data. These prior beliefs are then transformed into posterior beliefs based on the observed data. 

Bayesian modeling has two key components:

• A generalization function that results from integrating the predictions of all hypotheses weighted by their posterior probability
• The assumption that examples are sampled from the concept to be learned 

Bayesian modeling can be used in environmental health to inform the model with information from previous studies. For example, it can use previously estimated toxicities of certain pollutants. 

Bayesian modeling is a form of probabilistic inference, which is a method used to draw conclusions from uncertain evidence. According to these models, the human mind behaves like a capable data scientist or crime scene investigator when dealing with noisy and ambiguous data. 

- Bayesian Statistics vs. Machine Learning

The term “machine learning” is not precisely defined, and it can be considered to include Bayesian inference as a special case. Indeed, many machine learning methods are fit using Bayesian or approximately Bayesian methods.

Bayesian statistics focuses on updating beliefs in light of new data and explicitly handling uncertainty, while machine learning focuses on building predictive models that can learn complex patterns from large amounts of data.