How do we calculate $$ \begin{align}E[S]&=E\left[\sqrt{\frac{1}{n-1}\sum_i(x_i-\bar{x})^2}\right]\\&=E\left[\sqrt{\frac{1}{n-1}} \sqrt{\sum_ix_i^2-n\bar{x}^2}\right]. \end{align}$$
I am asking this question out of curiosity mostly because it is easy to show that $E[S^2]=\sigma^2$ but not so easy to calculate this.
$\endgroup$ 72 Answers
$\begingroup$When sampling from a normal population, the sample SD calculated from $n$ independent observations has expected value $$E(S)=\sigma \cdot \sqrt{ \frac{2}{n-1} } \cdot \frac{ \Gamma(n/2) }{ \Gamma( \frac{n-1}{2} ) },$$ where $\sigma$ is the population standard deviation. For non-normal populations it's hard to find an explicit formula for $E(S)$. But you can prove that for any population, the sample SD tends to underestimate the population SD.
See this article on stats.stackexchange.com for details.
$\endgroup$ 4 $\begingroup$$\newcommand{\e}{\operatorname{E}}$I've up-voted the answer by "grand_chat", but it omits some things, including (in the answer's present form) how to simplify the quotient of Gamma functions.
With any distribution with finite variance, you have $\e(S^2) = \sigma^2.$ But if I'm not mistaken, $\e(S)$ depends on the shape of the distribution. Maybe I'll say more about that later. I might start by looking at what it is for a Bernoulli distribution with $n=1,2,3.$
But assuming an i.i.d. sample from a normal distribution, and that it's already established that $$ (n-1) \frac{S^2}{\sigma^2} \sim \chi^2_{n-1}, $$ let us first recall the chi-square density (with $n-1$ degrees of freedom): $$ f(x)\,dx = \frac 1 {\Gamma(\frac{n-1} 2)} \left(\frac x 2\right)^{(n-3)/2} e^{-x/2} \, \left(\frac{dx} 2\right) \text{ for } x\ge0. $$ Therefore \begin{align} \e\left( \sqrt{n-1} \, \frac S \sigma \right)& = \int_0^\infty \sqrt x f(x) \,dx = \frac 1 {\Gamma\left( \frac{n-1} 2 \right)} \sqrt 2 \int_0^\infty \sqrt{\frac x 2 } \left( \frac x 2 \right)^{(n-3)/2} e^{-x/2} \, \left( \frac {dx} 2 \right) \\[10pt] & = \frac {\sqrt 2} {\Gamma\left( \frac{n-1} 2 \right)} \int_0^\infty \left( \frac x 2 \right)^{(n/2)-1} e^{-x/2} \, \left( \frac {dx} 2\right) = \frac{\sqrt 2}{\Gamma\left( \frac{n-1} 2 \right)} \cdot \Gamma \left( \frac n 2 \right) \\[15pt] \text{Therefore } \e S & = \sigma \sqrt{\frac 2 {n-1}} \cdot \frac{\Gamma\left( \frac n 2\right)}{\Gamma\left( \frac{n-1} 2 \right)}. \end{align}
I dislike piecewise definitions, but here I don't know off-hand any way (but maybe somebody does?) to avoid looking at two cases:
First assume $n$ is odd:
Then $$ \Gamma\left( \frac n 2 \right) = \frac{n-2}2\cdot\frac{n-4}2\cdots\cdots \frac 1 2 \Gamma\left( \frac 1 2 \right) = \frac{(n-1)!}{2^{n-1} \left( \frac{n-1} 2 \right)!} \Gamma\left( \frac 1 2 \right) = \frac{(n-1)!}{2^{n-1} \left( \frac{n-1} 2 \right)!} \sqrt \pi. $$ Then in the quotient of two Gamma functions above you will have $\displaystyle \frac{(n-1)!}{\left(\left( \frac{n-1} 2 \right)!\right)^2} = \binom { n-1 }{ \frac{n-1} 2 }.$
Do a similar thing if $n$ is even, working first on $\displaystyle\Gamma\left( \frac {n-1} 2\right).$
$\endgroup$
