Somebody else looking in
Why do we need sigma algebras to define probability spaces?

# Why do we need sigma algebras to define probability spaces?

This is an article quoted from Cross Validated.

https://stats.stackexchange.com/questions/199280/why-do-we-need-sigma-algebras-to-define-probability-spaces

# A theory of probability admitting all subsets of UNCONOUNTABLE sets will break mathematics

Consider this example. Suppose you have a unit square in $\mathbb{R}^2$, and you’re interested in the probability of randomly selecting a point that is a member of a specific set in the unit square. In many circumstances, this can be readily answered based on comparing areas of the different sets. For example, we can draw some circles, measure their areas, and then take the probability as the fraction of the square falling in the circle. Very simple.

But what if the area of the set of interest is not well-defined?

If the area is not well-defined, then we can reason two different but completely valid (in some sense) conclusions about what the area is. So we could have $P(A)=0$ on the one hand and $P(A)=0$ on the other, which implies 0=10=1. This breaks all of the math beyond repair. You can now prove 5<0 and several other incredible things. This isn’t too useful.

# 𝝈-algebras are the patch that fixes math

What is a 𝜎-algebra, precisely? It’s not that frightening. It’s just a definition of which sets may be considered events. Elements not in ℱ simply have no defined probability measure. 𝜎-algebras are the “patch” that lets us avoid some pathological behaviours of mathematics, namely non-measurable sets.

The three requirements of a 𝜎-field can be considered as consequences of what we would like to do with probability: A 𝜎-field is a set that has three properties:

1. Closure under countable unions;
2. Closure under countable intersections;
3. Closure under complements.

The countable unions and countable intersections components directly consequences of the non-measurable set issue. Closure under complements is a consequence of the Kolmogorov axioms: if $P(A)=\frac{2}{3}$, $P(A^c)$ ought to be 1/3. But without (3), $P(A^c)$ might be undefined. That would be strange. Closure under complements and the Kolmogorov axioms let us say things like $P(A\cup A^c)=P(A)+1-P(A)=1$.

Finally, We are considering events about $\Omega$, so we further require that $\Omega\in\mathcal{F}$.

# Good news: 𝝈-algebras are only strictly necessary for uncountable sets

But! There’s good news here, also. Or, at least, a way to skirt the issue. We only need 𝜎-algebras if we’re working in a set with uncountable cardinality. If we restrict ourselves to countable sets, then we can take $\mathcal{F}=2^\Omega$ the power set of $\Omega$ and we won’t have any of these problems because, for countable $\Omega$, $2^\Omega$ consists only of measurable sets. You’ll notice that some textbooks will actually commit a subtle sleight-of-hand here, and only consider countable sets when discussing probability spaces.

Additionally, in geometric problems in $\mathbb{R}^n$, it’s perfectly sufficient to only consider 𝜎-algebras composed of sets for which the $\mathcal{L}^n$ measure is defined. To ground this somewhat more firmly, $\mathcal{L}^n$ for $n=1,2,3$ corresponds to the usual notions of length, area and volume. So what I’m saying in the previous example is that the set needs to have a well-defined area for it to have a geometric probability assigned to it. And the reason is this: if we admit non-measurable sets, then we can end up in situations where we can assign probability 1 to some event based on some proof, and probability 0 to the same event based on some other proof.

But don’t let the connection to uncountable sets confuse you! A common misconception is that 𝜎-algebras are countable sets. They may be countable or uncountable. Consider this illustration: as before, we have a unit square. Define

\mathcal{F}=\text{All subsets of the unit sqaure with defined }\mathcal{L}^2 \text{ measure}.

You can draw a square $B$ with side length $s$ for all $s\in(0,1)$, and with one corner at $(0,0)$. It should be clear that this square is a subset of the unit square. Moreover, all of these squares have a defined area, so these squares are elements of $\mathcal{F}$. But it should also be clear that there are uncountably many squares $B$: the number of such squares is uncountable, and each square has defined the Lebesgue measure.

So as a practical matter, simply making that observation is often enough to make the observation that you only consider Lebesgue-measurable sets to gain headway against the problem of interest.

# But wait, what’s a non-measurable set?

I’m afraid I can only shed a little bit of light on this myself. But the Banach-Tarski paradox (sometimes the “sun and pea” paradox) can help us some:

Given a solid ball in 3‑dimensional space, there exists a decomposition of the ball into a finite number of disjoint subsets, which can then be put back together in a different way to yield two identical copies of the original ball. Indeed, the reassembly process involves only moving the pieces around and rotating them, without changing their shape. However, the pieces themselves are not “solids” in the usual sense, but infinite scatterings of points. The reconstruction can work with as few as five pieces.
A stronger form of the theorem implies that given any two “reasonable” solid objects (such as a small ball and a huge ball), either one can be reassembled into the other. This is often stated informally as “a pea can be chopped up and reassembled into the Sun” and called the “pea and the Sun paradox”.1

So if you’re working with probabilities in $\mathbb{R}^3$ and you’re using the geometric probability measure (the ratio of volumes), you want to work out the probability of some event. But you’ll struggle to define that probability precisely because you can rearrange the sets of your space to change volumes! If probability depends on volume, and you can change the volume of the set to be the size of the sun or the size of a pea, then the probability will also change. So no event will have a single probability ascribed to it. Even worse, you can rearrange $S\in\Omega$ such that the volume of $S$ has $V(S)>V(\Omega)$, which implies that the geometric probability measure reports a probability $P(S)>1$, in flagrant violation of the Kolmogorov axioms which require that probability has measure 1.

To resolve this paradox, one could make one of four concessions:

1. The volume of a set might change when it is rotated;
2. The volume of the union of two disjoint sets might be different from the sum of their volumes;
3. The axioms of Zermelo–Fraenkel set theory with the axiom of Choice (ZFC) might have to be altered;
4. Some sets might be tagged “non-measurable”, and one would need to check whether a set is “measurable” before talking about its volume.

Option (1) doesn’t help us define probabilities, so it’s out. Option (2) violates the second Kolmogorov axiom, so it’s out. Option (3) seems like a terrible idea because ZFC fixes so many more problems than it creates. But option (4) seems attractive: if we develop a theory of what is and is not measurable, then we will have well-defined probabilities in this problem! This brings us back to measure theory, and our friend the 𝜎-algebra.

## Some FAQs

### Why Sigma algebra is needed?

𝝈-algebras are the patch that fixes math, otherwise, some unpredictable outcomes may occur.

### What is the difference between algebra and sigma-algebra?

Algebras only require that they be closed under pairwise unions while σ-algebras must be closed under countably infinite unions.

### How do you prove something is a sigma-algebra?

Closure under countable unions;
Closure under countable intersections;
Closure under complements.