Here's a brief answer to the question at the bottom (to back up the graphical intuition). It only uses Calculus I, and it's a nice proof to see at least once, I think.
The basis functions are orthogonal if any pair of them has inner product 0. The inner product of $f(x)$ and $g(x)$ is
\begin{equation}\int_{-\pi}^\pi f(x)g(x)\mathrm{dx}\end{equation}
(I choose the limits $-\pi$ and $\pi$ to work with $\sin(nx)$ to make things a little cleaner. With the original limits of 0 and 1, $\sin(mx)$ and $\cos(nx)$ aren't orthogonal).
First, let's consider the inner product of $f(x)=\sin(mx)$ and $g(x)=\cos(nx)$. Letting $h(x) = f(x)g(x) = \sin(mx)\cos(nx)$, this inner product is
\begin{equation}\int_{-\pi}^\pi h(x)\mathrm{dx} \end{equation}
Observe that
\begin{equation}h(-x) = \sin(-mx)\cos(-nx) = -\sin(mx)\cos(nx) = -h(x)\end{equation}
Using this, we rewrite the inner product as
\begin{equation}\int_{-\pi}^0 h(x)\mathrm{dx}+\int_{0}^\pi h(x)\mathrm{dx} = \int_{\pi}^0 h(x')\mathrm{dx'}+\int_{0}^\pi h(x)\mathrm{dx}\end{equation}
(where we used the subsitution $x'=-x$ and $dx'=-dx$)
\begin{equation}=-\int_{0}^\pi h(x')\mathrm{dx'}+\int_{0}^\pi h(x)\mathrm{dx}=0\end{equation}
So that's nice, then $\sin(mx)$ and $\cos(nx)$ are orthogonal.
What about $f(x) = \sin(mx)$ and $g(x) = \sin(nx)$, where $m\neq n$ (since we only care about inner products of pairs of \emph{distinct} functions)? We make use of the following identity:
\begin{equation} \sin(a)\sin(b) = \frac{\cos(a-b) - \cos(a+b)}{2}\end{equation}
This identity is based on a set of trig identities (which many of us will remember from high school) which is proved geometrically here, for the enthusiastic reader to review: http://www.themathpage.com/atrig/sum-proof.htm
Anyways, with this identity in mind, the inner product of $\sin(mx)$ and $\sin(nx)$ becomes
\begin{equation}\int_{-\pi}^{\pi}\sin(mx)\sin(nx)\mathrm{dx}=\frac{1}{2}\int_{-\pi}^{\pi} \cos((m-n)x) - \cos((m+n)x)\mathrm{dx}\end{equation}
\begin{equation}=\frac{1}{2}\left(\frac{\sin(\pi(m-n))}{m-n} - \frac{\sin(\pi(m+n))}{m+n}
-\frac{\sin(-\pi(m-n))}{m-n} + \frac{\sin(-\pi(m+n))}{m+n}\right)\end{equation}
Since $m-n$ is an integer, $\sin(\pi(m-n))=\sin(-\pi(m-n))=0$, so the above expression is equal to 0.
Don't worry about division by zero, remember that $m\neq n$ and $m,n>0$. A pretty similar procedure will yield $\int_{-\pi}^{\pi}\cos(mx)\cos(nx)\mathrm{dx}=0$ (hint: use $\cos(a)\cos(b) = \frac{1}{2}(\cos(a-b)+\cos(a+b))$, whence all the basis functions are orthogonal.
taoy1
I think almost the same as the above answer to prove the bases orthogonal. But I was stuck at the assumption that the domain of the inner product of $f(x)$ and $g(x)$ is $(-\pi, \pi)$ or $(-\infty, \infty)$ or some original domain of $f(x)$ and $g(x)$.
As mentioned above, the basis functions $f(x)$ and $g(x)$ can be any $cos(nx), n\in$$\mathbb{N}$ or $sin(nx), n\in$$\mathbb{N}$
Could anybody answer this question?
johnryan
Well the domain may be $(-L,L)$ for any $L>0$ and the argument works, except that $\sin(nx)$ and $\cos(nx)$ become
\begin{equation}
\sin(\frac{n\pi x}{L}) \end{equation}
and
\begin{equation}
\cos(\frac{n\pi x}{L}) \end{equation}
Notice that the case $L=\pi$ makes the math cleaner.
johnryan
But if I chose $L=1$ and just used $\sin(nx)$ and $\cos(mx)$ for any natural $m,n$, they are NOT orthogonal: https://www.wolframalpha.com/input/?i=integral+of+sin(nx)sin(mx)+from+-1+to+1
As for $(0,L)$, the functions $\sin(\frac{n\pi x}{L})$ form an orthogonal basis. More on that here: https://en.wikipedia.org/wiki/Fourier_sine_and_cosine_series
keenan
You have to integrate over at least a whole period (and in general, an integer number of periods) in order to get orthogonality. This fact is suggested in the lower-right image: basically it's symmetries in the product of two bases that result in exact cancellation. So, if you don't cover at least a whole period, you don't see this symmetry. Likewise, if you cover a whole period plus a fractional period, you'll again have extra "junk" that doesn't get cancelled.
On the other hand, this symmetry idea suggests that there may be different collections of functions, other than the sinusoids, that provide a basis for an interval of the real line. (Can you think of any?)
Here's a brief answer to the question at the bottom (to back up the graphical intuition). It only uses Calculus I, and it's a nice proof to see at least once, I think.
The basis functions are orthogonal if any pair of them has inner product 0. The inner product of $f(x)$ and $g(x)$ is \begin{equation}\int_{-\pi}^\pi f(x)g(x)\mathrm{dx}\end{equation} (I choose the limits $-\pi$ and $\pi$ to work with $\sin(nx)$ to make things a little cleaner. With the original limits of 0 and 1, $\sin(mx)$ and $\cos(nx)$ aren't orthogonal).
First, let's consider the inner product of $f(x)=\sin(mx)$ and $g(x)=\cos(nx)$. Letting $h(x) = f(x)g(x) = \sin(mx)\cos(nx)$, this inner product is \begin{equation}\int_{-\pi}^\pi h(x)\mathrm{dx} \end{equation} Observe that \begin{equation}h(-x) = \sin(-mx)\cos(-nx) = -\sin(mx)\cos(nx) = -h(x)\end{equation} Using this, we rewrite the inner product as \begin{equation}\int_{-\pi}^0 h(x)\mathrm{dx}+\int_{0}^\pi h(x)\mathrm{dx} = \int_{\pi}^0 h(x')\mathrm{dx'}+\int_{0}^\pi h(x)\mathrm{dx}\end{equation} (where we used the subsitution $x'=-x$ and $dx'=-dx$)
\begin{equation}=-\int_{0}^\pi h(x')\mathrm{dx'}+\int_{0}^\pi h(x)\mathrm{dx}=0\end{equation} So that's nice, then $\sin(mx)$ and $\cos(nx)$ are orthogonal.
What about $f(x) = \sin(mx)$ and $g(x) = \sin(nx)$, where $m\neq n$ (since we only care about inner products of pairs of \emph{distinct} functions)? We make use of the following identity:
\begin{equation} \sin(a)\sin(b) = \frac{\cos(a-b) - \cos(a+b)}{2}\end{equation} This identity is based on a set of trig identities (which many of us will remember from high school) which is proved geometrically here, for the enthusiastic reader to review: http://www.themathpage.com/atrig/sum-proof.htm
Anyways, with this identity in mind, the inner product of $\sin(mx)$ and $\sin(nx)$ becomes \begin{equation}\int_{-\pi}^{\pi}\sin(mx)\sin(nx)\mathrm{dx}=\frac{1}{2}\int_{-\pi}^{\pi} \cos((m-n)x) - \cos((m+n)x)\mathrm{dx}\end{equation}
\begin{equation}=\frac{1}{2}\left(\frac{\sin((m-n)x)}{m-n} - \frac{\sin((m+n)x)}{m+n}\right)\Big|_{-\pi}^{\pi}\end{equation}
\begin{equation}=\frac{1}{2}\left(\frac{\sin(\pi(m-n))}{m-n} - \frac{\sin(\pi(m+n))}{m+n} -\frac{\sin(-\pi(m-n))}{m-n} + \frac{\sin(-\pi(m+n))}{m+n}\right)\end{equation} Since $m-n$ is an integer, $\sin(\pi(m-n))=\sin(-\pi(m-n))=0$, so the above expression is equal to 0.
Don't worry about division by zero, remember that $m\neq n$ and $m,n>0$. A pretty similar procedure will yield $\int_{-\pi}^{\pi}\cos(mx)\cos(nx)\mathrm{dx}=0$ (hint: use $\cos(a)\cos(b) = \frac{1}{2}(\cos(a-b)+\cos(a+b))$, whence all the basis functions are orthogonal.
I think almost the same as the above answer to prove the bases orthogonal. But I was stuck at the assumption that the domain of the inner product of $f(x)$ and $g(x)$ is $(-\pi, \pi)$ or $(-\infty, \infty)$ or some original domain of $f(x)$ and $g(x)$.
As mentioned above, the basis functions $f(x)$ and $g(x)$ can be any $cos(nx), n\in$$\mathbb{N}$ or $sin(nx), n\in$$\mathbb{N}$
Could anybody answer this question?
Well the domain may be $(-L,L)$ for any $L>0$ and the argument works, except that $\sin(nx)$ and $\cos(nx)$ become \begin{equation} \sin(\frac{n\pi x}{L}) \end{equation} and \begin{equation} \cos(\frac{n\pi x}{L}) \end{equation} Notice that the case $L=\pi$ makes the math cleaner.
But if I chose $L=1$ and just used $\sin(nx)$ and $\cos(mx)$ for any natural $m,n$, they are NOT orthogonal: https://www.wolframalpha.com/input/?i=integral+of+sin(nx)sin(mx)+from+-1+to+1
As for $(0,L)$, the functions $\sin(\frac{n\pi x}{L})$ form an orthogonal basis. More on that here: https://en.wikipedia.org/wiki/Fourier_sine_and_cosine_series
You have to integrate over at least a whole period (and in general, an integer number of periods) in order to get orthogonality. This fact is suggested in the lower-right image: basically it's symmetries in the product of two bases that result in exact cancellation. So, if you don't cover at least a whole period, you don't see this symmetry. Likewise, if you cover a whole period plus a fractional period, you'll again have extra "junk" that doesn't get cancelled.
On the other hand, this symmetry idea suggests that there may be different collections of functions, other than the sinusoids, that provide a basis for an interval of the real line. (Can you think of any?)