In probability theory, a sub-Gaussian distribution, the distribution of a sub-Gaussian random variable, is a probability distribution with strong tail decay. More specifically, the tails of a sub-Gaussian distribution are dominated by (i.e. decay at least as fast as) the tails of a Gaussian. This property gives sub-Gaussian distributions their name.

Formally, the probability distribution of a random variable ${\displaystyle X}$ is called sub-Gaussian if there is a positive constant C such that for every ${\displaystyle t\geq 0}$,

${\textstyle \operatorname {P} (|X|\geq t)\leq 2\exp {(-t^{2}/C^{2})}}$.

Alternatively, a random variable is considered sub-Gaussian if its distribution function is upper bounded (up to a constant) by the distribution function of a Gaussian. Specifically, we say that ${\displaystyle X}$ is sub-Gaussian if for all ${\displaystyle s\geq 0}$ we have that:

${\displaystyle P(|X|\geq s)\leq cP(|Z|\geq s),}$

where ${\displaystyle c\geq 0}$ is constant and ${\displaystyle Z}$ is a mean zero Gaussian random variable.^{[1]: Theorem 2.6}

Definitions

The sub-Gaussian norm of ${\displaystyle X}$, denoted as ${\displaystyle \Vert X\Vert _{gauss}}$, is defined by

$${\displaystyle \Vert X\Vert _{gauss}=\inf \left\{c>0:\operatorname {E} \left[\exp {\left({\frac {X^{2}}{c^{2}}}\right)}\right]\leq 2\right\},}$$

which is the Orlicz norm of ${\displaystyle X}$ generated by the Orlicz function ${\displaystyle \Phi (u)=e^{u^{2}}-1.}$ By condition ${\displaystyle (2)}$ below, sub-Gaussian random variables can be characterized as those random variables with finite sub-Gaussian norm.

Sub-Gaussian properties

Let ${\displaystyle X}$ be a random variable. The following conditions are equivalent:

1. ${\displaystyle \operatorname {P} (|X|\geq t)\leq 2\exp {(-t^{2}/K_{1}^{2})}}$ for all ${\displaystyle t\geq 0}$, where ${\displaystyle K_{1}}$ is a positive constant;
2. ${\displaystyle \operatorname {E} [\exp {(X^{2}/K_{2}^{2})}]\leq 2}$, where ${\displaystyle K_{2}}$ is a positive constant;
3. ${\displaystyle \operatorname {E} |X|^{p}\leq 2K_{3}^{p}\Gamma \left({\frac {p}{2}}+1\right)}$ for all ${\displaystyle p\geq 1}$, where ${\displaystyle K_{3}}$ is a positive constant.

Proof. ${\displaystyle (1)\implies (3)}$ By the layer cake representation,

$${\displaystyle {\begin{aligned}\operatorname {E} |X|^{p}&=\int _{0}^{\infty }\operatorname {P} (|X|^{p}\geq t)dt\\&=\int _{0}^{\infty }pt^{p-1}\operatorname {P} (|X|\geq t)dt\\&\leq 2\int _{0}^{\infty }pt^{p-1}\exp \left(-{\frac {t^{2}}{K_{1}^{2}}}\right)dt\\\end{aligned}}}$$

After a change of variables ${\displaystyle u=t^{2}/K_{1}^{2}}$, we find that

$${\displaystyle {\begin{aligned}\operatorname {E} |X|^{p}&\leq 2K_{1}^{p}{\frac {p}{2}}\int _{0}^{\infty }u^{{\frac {p}{2}}-1}e^{-u}du\\&=2K_{1}^{p}{\frac {p}{2}}\Gamma \left({\frac {p}{2}}\right)\\&=2K_{1}^{p}\Gamma \left({\frac {p}{2}}+1\right).\end{aligned}}}$$

${\displaystyle (3)\implies (2)}$ Using the Taylor series for ${\displaystyle e^{x}}$:

$${\displaystyle e^{x}=1+\sum _{p=1}^{\infty }{\frac {x^{p}}{p!}},}$$

we obtain that

$${\displaystyle {\begin{aligned}\operatorname {E} [\exp {(\lambda X^{2})}]&=1+\sum _{p=1}^{\infty }{\frac {\lambda ^{p}\operatorname {E} {[X^{2p}]}}{p!}}\\&\leq 1+\sum _{p=1}^{\infty }{\frac {2\lambda ^{p}K_{3}^{2p}\Gamma \left(p+1\right)}{p!}}\\&=1+2\sum _{p=1}^{\infty }\lambda ^{p}K_{3}^{2p}\\&=2\sum _{p=0}^{\infty }\lambda ^{p}K_{3}^{2p}-1\\&={\frac {2}{1-\lambda K_{3}^{2}}}-1\quad {\text{for }}\lambda K_{3}^{2}<1,\end{aligned}}}$$

which is less than or equal to ${\displaystyle 2}$ for ${\displaystyle \lambda \leq {\frac {1}{3K_{3}^{2}}}}$. Take ${\displaystyle K_{2}\geq 3^{\frac {1}{2}}K_{3}}$, then

$${\displaystyle \operatorname {E} [\exp {(X^{2}/K_{2}^{2})}]\leq 2.}$$

${\displaystyle (2)\implies (1)}$ By Markov's inequality,

$${\displaystyle \operatorname {P} (|X|\geq t)=\operatorname {P} \left(\exp \left({\frac {X^{2}}{K_{2}^{2}}}\right)\geq \exp \left({\frac {t^{2}}{K_{2}^{2}}}\right)\right)\leq {\frac {\operatorname {E} [\exp {(X^{2}/K_{2}^{2})}]}{\exp \left({\frac {t^{2}}{K_{2}^{2}}}\right)}}\leq 2\exp \left(-{\frac {t^{2}}{K_{2}^{2}}}\right).}$$

More equivalent definitions

The following properties are equivalent:

• The distribution of ${\displaystyle X}$ is sub-Gaussian.
• Laplace transform condition: for some B, b > 0, ${\displaystyle \operatorname {E} e^{\lambda (X-\operatorname {E} [X])}\leq Be^{\lambda ^{2}b}}$ holds for all ${\displaystyle \lambda }$.
• Moment condition: for some K > 0, ${\displaystyle \operatorname {E} |X|^{p}\leq K^{p}p^{p/2}}$ for all ${\displaystyle p\geq 1}$.
• Moment generating function condition: for some ${\displaystyle L>0}$, ${\displaystyle \operatorname {E} [\exp(\lambda ^{2}X^{2})]\leq \exp(L^{2}\lambda ^{2})}$ for all ${\displaystyle \lambda }$ such that ${\displaystyle |\lambda |\leq {\frac {1}{L}}}$. ^[2]
• Union bound condition: for some c > 0, ${\displaystyle \ \operatorname {E} [\max\{|X_{1}-\operatorname {E} [X]|,\ldots ,|X_{n}-\operatorname {E} [X]|\}]\leq c{\sqrt {\log n}}}$ for all n > c, where ${\displaystyle X_{1},\ldots ,X_{n}}$ are i.i.d copies of X.

Examples

A standard normal random variable ${\displaystyle X\sim N(0,1)}$ is a sub-Gaussian random variable.

Let ${\displaystyle X}$ be a random variable with symmetric Bernoulli distribution (or Rademacher distribution). That is, ${\displaystyle X}$ takes values ${\displaystyle -1}$ and ${\displaystyle 1}$ with probabilities ${\displaystyle 1/2}$ each. Since ${\displaystyle X^{2}=1}$, it follows that

$${\displaystyle \Vert X\Vert _{gauss}=\inf \left\{c>0:\operatorname {E} \left[\exp {\left({\frac {X^{2}}{c^{2}}}\right)}\right]\leq 2\right\}=\inf \left\{c>0:\operatorname {E} \left[\exp {\left({\frac {1}{c^{2}}}\right)}\right]\leq 2\right\}={\frac {1}{\sqrt {\ln 2}}},}$$

and hence ${\displaystyle X}$ is a sub-Gaussian random variable.

Maximum of Sub-Gaussian Random Variables

Consider a finite collection of subgaussian random variables, X₁, ..., X_n, with corresponding subgaussian parameters ${\displaystyle \sigma }$. The random variable M_n = max(X₁, ..., X_n) represents the maximum of this collection. The expectation ${\displaystyle M_{n}}$ can be bounded above by ${\displaystyle {\sqrt {2\sigma ^{2}\log n}}}$. Note that no independence assumption is needed to form this bound.^[1]

See also

• Platykurtic distribution

Notes

References

• Kahane, J.P. (1960). "Propriétés locales des fonctions à séries de Fourier aléatoires". Studia Mathematica. 19: 1–25. doi:10.4064/sm-19-1-1-25.
• Buldygin, V.V.; Kozachenko, Yu.V. (1980). "Sub-Gaussian random variables". Ukrainian Mathematical Journal. 32 (6): 483–489. doi:10.1007/BF01087176.
• Ledoux, Michel; Talagrand, Michel (1991). Probability in Banach Spaces. Springer-Verlag.
• Stromberg, K.R. (1994). Probability for Analysts. Chapman & Hall/CRC.
• Litvak, A.E.; Pajor, A.; Rudelson, M.; Tomczak-Jaegermann, N. (2005). "Smallest singular value of random matrices and geometry of random polytopes" (PDF). Advances in Mathematics. 195 (2): 491–523. doi:10.1016/j.aim.2004.08.004.
• Rudelson, Mark; Vershynin, Roman (2010). "Non-asymptotic theory of random matrices: extreme singular values". Proceedings of the International Congress of Mathematicians 2010. pp. 1576–1602. arXiv:1003.2990. doi:10.1142/9789814324359_0111.
• Rivasplata, O. (2012). "Subgaussian random variables: An expository note" (PDF). Unpublished.
• Vershynin, R. (2018). "High-dimensional probability: An introduction with applications in data science" (PDF). Volume 47 of Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge.
• Zajkowskim, K. (2020). "On norms in some class of exponential type Orlicz spaces of random variables". Positivity. An International Mathematics Journal Devoted to Theory and Applications of Positivity. 24(5): 1231--1240. arXiv:1709.02970. doi.org/10.1007/s11117-019-00729-6.