Briefly, both the law of large numbers and central limit theorem are about many independent samples from same distribution.
The LLN tells us two things:
- The average of many independent samples is (with high probability) close to the mean of the underlying distribution;
- This density histogram of many independent samples is (with high probability) close to the graph of the density of the underlying distribution.
The central limit theorem says that the sum or average of many independent copies of a random variable is approximately a normal random variable. So basically, the CLT gives precise values for the mean and standard deviation of the normal variable.
The mathematics of the LLN says that the average of a lot of independent samples from a random variable will almost certainly approach the mean of the variable, but they can’t tell us if the tool or experiment is producing data worth averaging.
Suppose X1, X2, . . . , Xn are independent random variables with the same underlying
distribution; we say that the Xi are independent and identically-distributed,
or i.i.d.
Let now Xn be the average of X1, . . . , Xn:
Note that Xn is itself a random variable. The law of large numbers and central limit
theorem tell us about the value and distribution of Xn, respectively:
- LLN = As n grows, the probability that Xn is close to µ goes to 1.
- CLT = As n grows, the distribution of Xn converges to the normal distribution N(µ, σ2/n).
So far, by reading both the LLN and the CLT definitions, we may wonder:
“The Central limit Theorem states that when sample size tends to infinity, the sample mean will be normally distributed. The Law of Large Number states that when sample size tends to infinity, the sample mean equals to population mean.
Is the two statements contradictory?“
The answer would be that no, the two statements are not contradictory.
The Central Limit Theorem tells us that as the sample size tends to infinity, the center of the distribution of sample means approaches the normal distribution. This is a statement about, indeed, the shape of the distribution.
A normal distribution is bell shaped, so the shape of the distribution of sample means begins to look bell shaped as the sample size increases.
On the other hand, the Law of Large Numbers tells us where the center (maximum point) of the bell is located. Again, as the sample size approaches infinity, the center of the distribution of the sample means becomes very close to the population mean.
So is incorrect to say CLT and LLN aren’t related; they actually state two different aspects of the same thing.
Rambling things 