Cesàro's lemma

Proof Let $(V, \left\| \cdot\right\|)$ denote the underlying normed vector space and $\ell \in V$ denote the limit of $(u_{n})$. Let $\varepsilon > 0$ be fixed. Since $(u_{n})$ converges to $\ell$, there exists $N_{\varepsilon} \geq 0$ such that $\left\| u_{n} - \ell\right\| \leq \varepsilon / 2$ for all $n \geq N_{\varepsilon}$. For all $n \geq N_{\varepsilon}$, we have

$$\left\| \frac{1}{n} \sum_{k = 1}^{n} u_{k} - \ell\right\| \leq \frac{1}{n} \sum_{k = 1}^{N_{\varepsilon}} \left\| u_{k} - \ell\right\| + \frac{1}{n} \sum_{k = N_{\varepsilon} + 1}^{n} \left\| u_{k} - \ell\right\| \leq \frac{S_{\varepsilon}}{n} + \frac{1}{2} \frac{n - N_{\varepsilon}}{n} \varepsilon,$$

where $S_{\varepsilon} = \sum_{k = 1}^{N_{\varepsilon}} \left\| u_{k} - \ell\right\|$ is independent of $n$. Since $S_{\varepsilon} / n \to 0$, there exists $N_{\varepsilon}' \geq 1$ such that $S_{\varepsilon} / n \leq \varepsilon / 2$ for all $n \geq N_{\varepsilon}'$. Observing that $(n - N_{\varepsilon}) / n \leq 1$, we conclude that $\left\| \frac{1}{n} \sum_{k = 1}^{n} u_{k} - \ell\right\| \leq \varepsilon$ for all $n \geq \max(N_{\varepsilon}, N_{\varepsilon}')$. $\blacksquare$

Proof Suppose $(u_{n})$ converges to $+\infty$. For all $A > 0$, there exists $N_{A} \geq 1$ such that $u_{n} \geq 4 A$ for all $n \geq N_{A}$. Then,

$$\frac{1}{n} \sum_{k = 1}^{n} u_{k} = \frac{1}{n} \sum_{k = 1}^{N_{A}} u_{k} + \frac{1}{n} \sum_{k = N_{A}+1}^{n} u_{k} \geq \frac{S_{A}}{n} + 4 \frac{n - N_{A}}{n} A,$$

where $S_{A} = \sum_{k = 1}^{N} u_{k}$ is independent on $n$. Since $S_{A} / n \to 0$, there exists $N_{A}' \geq 1$ such that $S_{A} / n \geq -A$ for all $n \geq N_{A}'$. Since $(n - N_{A}) / n \to 1$, there exists $N_{A}'' \geq 1$ such that $(n - N_{A}) / n \geq 1/2$ for all $n \geq N_{A}''$. Choosing $n \geq \max(N_{A}, N_{A}', N_{A}'')$, we find $\frac{1}{n} \sum_{k = 1}^{n} u_{k} \geq A$, so the averages of $(u_{n})$ converge to $+\infty$.

The same reasoning applies if $(u_{n})$ converges to $-\infty$. $\blacksquare$

Proof By the monotone convergence theorem for real-valued sequences, we infer that $(u_{n})$ converges to some $\ell \in \overline{\mathbb{ R}}$. By Cesàro's lemma, the averages of $(u_{n})$ also converge to $\ell$. $\blacksquare$

Proof Introduce $v_{1} = u_{1}$ and $v_{n} = u_{n} - u_{n-1}$. Then rewrite

$$\begin{align*} u_{n} - \frac{1}{n} \sum_{k = 1}^{n} u_{k} &= \sum_{j = 1}^{n} v_{j} - \frac{1}{n} \sum_{k = 1}^{n} \sum_{j = 1}^{k} v_{j} \\ &= \sum_{j = 1}^{n} v_{j} - \frac{1}{n} \sum_{j = 1}^{n} \sum_{k = j}^{n} v_{j} \\ &= \sum_{j = 1}^{n} v_{j} - \frac{1}{n} \sum_{j = 1}^{n} (n - j + 1) v_{j} \\ &= \frac{1}{n} \sum_{j = 1}^{n} (j - 1) v_{j}. \end{align*}$$

We have assumed $(j - 1) v_{j} \to 0$, so by Cesàro's lemma, we infer $\frac{1}{n} \sum_{j = 1}^{n} (j - 1) v_{j} \to 0$.

Let $\varepsilon > 0$. By the above, there exists $N_{\varepsilon} \geq 1$ such that $\left\| u_{n} - \frac{1}{n} \sum_{k = 1}^{n} u_{k}\right\| \leq \varepsilon / 2$ for all $n \geq N_{\varepsilon}$. Since the averages of $(u_{n})$ converge to $\ell$, there exists $N_{\varepsilon}'$ such that $\left\| \frac{1}{n} \sum_{k = 1}^{n} u_{k} - \ell\right\| \leq \varepsilon / 2$ for all $n \geq N_{\varepsilon}'$. We conclude that

$$\left\| u_{n} - \ell\right\| \leq \left\| u_{n} - \frac{1}{n} \sum_{k = 1}^{n} u_{k}\right\| + \left\| \frac{1}{n} \sum_{k = 1}^{n} u_{k} - \ell\right\| \leq \varepsilon$$

for all $n \geq \max(N_{\varepsilon}, N_{\varepsilon}')$. $\blacksquare$

Proof Define $v_{n} = u_{n+1} - u_{n}$. Let $m > n > 1$. We compute

$$\begin{align*} \frac{\sum_{k = n+1}^{m} u_{k}}{m - n} - u_{n} &= \frac{1}{m - n} \sum_{k = n+1}^{m} (u_{k} - u_{n}) \\ &= \frac{1}{m - n} \sum_{k = n+1}^{m} \sum_{j = n}^{k - 1} (u_{j+1} - u_{j}) \\ &= \frac{1}{m - n} \sum_{k = n+1}^{m} \sum_{j = n}^{k - 1} v_{j} \\ &= \frac{1}{m - n} \sum_{j = n}^{m - 1} \sum_{k = j+1}^{m} v_{j} \\ &= \sum_{j = n}^{m - 1} \frac{m - j}{m - n} v_{j}. \end{align*}$$

Since $u_{n+1} - u_{n} = O(1/n)$, we infer that there exists $K > 0$ such that $v_{j} \leq K / j$. Therefore

$$\frac{\sum_{k = n+1}^{m} u_{k}}{m - n} - u_{n} \leq \sum_{j = n}^{m - 1} \frac{m - j}{m - n} \frac{K}{j} \leq K \sum_{j = n + 1}^{m - 1} \frac{1}{j} \leq K \ln\left(\frac{m - 1}{n}\right),$$

where we used $1/j \leq \ln(j) - \ln(j-1)$, and the fact that the term corresponding to $j = n$ is zero. Introduce $a_{n} = \frac{1}{n} \sum_{k = 1}^{n} u_{k}$ such that $(a_{n})$ converges to $\ell$. We now decompose

$$\begin{align*} \left\| u_{n} - \ell\right\| & \leq \left\| u_{n} - \frac{\sum_{k = n+1}^{m} u_{k}}{m - n}\right\| + \left\| \frac{\sum_{k = n+1}^{m} u_{k}}{m - n} - \ell\right\| \\ & \leq K \ln\left(\frac{m - 1}{n}\right) + \left\| \frac{m a_{m} - n a_{n}}{m - n} - \ell\right\| \\ & \leq K \ln\left(\frac{m - 1}{n}\right) + \frac{m}{m - n} \left\| a_{m} - \ell\right\| + \frac{n}{m - n} \left\| a_{n} - \ell\right\|. \end{align*}$$

Choose $m(n) = 1 + \left\lfloor \gamma n \right\rfloor$ for some $\gamma > 1$, so that the first term is bounded by $K \ln \gamma$. Now let $\varepsilon > 0$ and choose $\gamma_{\varepsilon}$ such that $K \ln \gamma_{\varepsilon} \leq \varepsilon / 2$. Let $\alpha_{\varepsilon} > 0$. Since $(a_{n}) \to \ell$, there exists $N_{\varepsilon}$ such that $\left\| a_{n} - \ell\right\| \leq \alpha_{\varepsilon} \varepsilon$ for all $n \geq N_{\varepsilon}$, and therefore

$$\left\| u_{n} - \ell\right\| \leq \left(\frac{1}{2} + \alpha_{\varepsilon} \frac{m(n) + n}{m(n) - n}\right) \varepsilon \leq \left(\frac{1}{2} + \alpha_{\varepsilon} \frac{(\gamma_{\varepsilon} + 1) n + 1}{(\gamma_{\varepsilon} - 1) n}\right) \varepsilon \leq \left(\frac{1}{2} + \alpha_{\varepsilon} \frac{\gamma_{\varepsilon} + 2}{\gamma_{\varepsilon} - 1}\right) \varepsilon,$$

where we used $n \geq 1$ to obtain the last inequality. Choosing $\alpha_{\varepsilon} = (\gamma_{\varepsilon} - 1) / (2 (\gamma_{\varepsilon} + 2))$, we reach $\left\| u_{n} - \ell\right\| \leq \varepsilon$ for all $n \geq N_{\varepsilon}$. $\blacksquare$