Issue Observed

different results of std in matlab and numpy

TLDR: Numpy

ddof = 0, divide N, biased estimator

ddof = 1, divide N-1, unbiased estimator

Bessel’s Correction

Degree of freedom

Number of values in the final calculation of a statistic that are free to vary
e.g in sample variance, the degree of freedom is n-1
cuz the sample mean is fixed, so the last value is fixed by the rest

The higher the degree of freedom, sample variance is more reliable

Notation

s2s^2: Unbiased sample variance
sn2s_n^2: Biased sample variance
μ\mu:population mean
x‾\overline{x}: sample mean

Biased Reason

Crux: xˉ\bar{x} and μ\mu bias
ez prove

∑i=1n(xi−xˉ)2≤∑i=1n(xi−μ)2\sum\limits_{i=1}^{n}(x_i-\bar{x})^2 \leq \sum\limits_{i=1}^{n}(x_i-\mu)^2

Unbiased Estimator Proof

E(xˉ)=μE(\bar{x})=\mu
E(sn2)=E(1n∑i=1n(xi−xˉ)2)=E(1n∑i=1n((xi−μ)−(xˉ−μ)2))=E(1n∑i=1n((xi−μ)2−2(xi−μ)(xˉ−μ)+(xˉ−μ)2))=E(1n∑i=1n(xi−μ)2+1n∑i=1n(−2xi+2μ+xˉ−μ)(xˉ−μ))=E(1n∑i=1n(xi−μ)2+(xˉ−μ)n∑i=1n(−2xi+μ+xˉ))=E(1n∑i=1n(xi−μ)2−(xˉ−μ)2)=E(s2)−E((xˉ−μ)2)=σ2−Var(xˉ)=σ2−Var(1n∑i=1nxi)=σ2−1n2∑i=1nVar(xi)=σ2−1n2nσ2=n−1nσ2=n−1nE(s2)\begin{aligned} E\left(s_n^2\right) & =E\left(\frac{1}{n} \sum_{i=1}^n\left(x_i-\bar{x}\right)^2 \right) \\ & =E\left(\frac{1}{n} \sum_{i=1}^n\left(\left(x_i-\mu\right)-(\bar{x}-\mu)^2\right)\right) \\ & =E\left(\frac{1}{n} \sum_{i=1}^n\left(\left(x_i-\mu\right)^2-2\left(x_i-\mu\right)(\bar{x}-\mu)+(\bar{x}-\mu)^2\right)\right)\\ & =E\left(\frac{1}{n} \sum_{i=1}^n\left(x_i-\mu\right)^2+\frac{1}{n}\sum_{i=1}^n(-2x_i+2\mu+\bar{x}-\mu)(\bar{x}-\mu)\right) \\ & =E\left(\frac{1}{n} \sum_{i=1}^n\left(x_i-\mu\right)^2+\frac{(\bar{x}-\mu)}{n}\sum_{i=1}^n(-2x_i+\mu+\bar{x})\right) \\ & =E\left(\frac{1}{n} \sum_{i=1}^n\left(x_i-\mu\right)^2-(\bar{x}-\mu)^2\right) \\ & =E\left(s^2\right)-E\left((\bar{x}-\mu)^2\right) \\ & =\sigma^2-Var(\bar{x})\\ & =\sigma^2-Var(\frac{1}{n}\sum_{i=1}^n x_i)\\ & =\sigma^2-\frac{1}{n^2}\sum_{i=1}^n Var(x_i)\\ & = \sigma^2 - \frac{1}{n^2}n\sigma^2\\ & = \frac{n-1}{n}\sigma^2 \\ & = \frac{n-1}{n}E(s^2) \end{aligned}
E(s2)=nn−1E(sn2)E(s^2)=\frac{n}{n-1}E(s_n^2)
sn2=1n∑i=1n(xi−xˉ)2s_n^2=\frac{1}{n}\sum\limits_{i=1}^n \left(x_i-\bar{x}\right)^2
s2=1n−1∑i=1n(xi−xˉ)2s^2 = \frac{1}{n-1}\sum\limits_{i=1}^n \left(x_i-\bar{x}\right)^2