Research IV) Axioms and frequency

Given the definitions in the previous research, let’s now study how the relative frequency can be linked to the probability axioms.

We’ve already said how the interpretation of Bayes’s theorem depends on the interpretation of probability given to the terms of study. In particular, from the frequentist point of view and from the Bayesian point of view, we’ll realize how the probabilistic theorem of Bayes will also work with relative frequency, as this is one of the main known approaches to the probability theory.

Frequentist probability defines the probability of an event as the limit of its relative frequency, in a large number of trials. (The limit of a sequence is the value that the terms of a sequence “tend to”.)

In the frequentist interpretation, probabilities are discussed only when dealing with well-defined random samples. The set of all possible outcomes of a random experiment is called the sample space of the experiment, while an event is defined as a particular subset of the considered sample space.
For any given event, only one of two possibilities may hold: it occurs or it does not.
The relative frequency of occurrence of an event, observed in a number of repetitions of the experiment, is a measure of the probability of that event. This is the core conception of probability in the frequentist interpretation.

To give an example, if  is the total number of trials and  is the number of trials where the event  occurred, the probability P(x) of the event occurring will be approximated by the relative frequency as follows:
As the number of trials is increased, the relative frequency will tend to become a better approximation of a “true frequency”, eventually converging exactly to the true probability:So, for frequentist probability it doesn’t make any sense to associate a probability distribution with a parameter. This means that when we compute a confidence interval, we interpret the ends of the confidence interval as random variables, and we talk about “the probability that the interval includes the true parameter”, rather than “the probability that the parameter is inside the confidence interval”.

On the other hand, in the Bayesian approach, we interpret probability distributions as quantifying our uncertainty about the world. This means that we can now meaningfully talk about probability distributions of parameters, since even though the parameter is fixed, our knowledge of its true value may be limited.

For example, from the Bernoulli distribution with parameter p we can define the sample success rate p^ such that:we can now invert the probability distribution f(p|p^) using Bayes’ law, to giveAnother example is shown below:

Questo slideshow richiede JavaScript.

By Bayesian interpretation of probability, we have to introduce the prior distribution into our analysis that actually reflects the belief about the value of p before seeing the actual values of the Xi.
This is were Bayesian and frequentist approaches collide; the role of the prior is often criticised in the frequentist approach, since it’s argued that it introduces subjectivity into the otherwise object world of probability, whereas in the Bayesian approach it’s no more about talking of confidence intervals, but instead of credible intervals, which have a more natural interpretation.

It might also be good to mention that the gap between the frequentist and Bayesian approaches is not nearly as great on a practical level: any frequentist method that produces useful and self-consistent results can generally be given a Bayesian interpretation, and vice versa. Both interpretations are useful in applications, and which is more useful depends on the situation.

Glossary:

Frequency:= number of occurrences of a repeating event per unit time.
synonyms: density, number, prevalence, recurrence, regularity, repetition.

Bayesian:= an interpretation of the concept of probability, in which, instead of frequency or propensity of some phenomenon, probability is interpreted as reasonable expectation.
synonyms: —

Subjective:= a subject’s personal perspective
synonyms: abstract, biased, intuitive, personal.

Objective:= a mathematical approach with which certain measures are invariant relative to observer
synonyms: detached, disinterested, unbiased, equitable.

• researches