A very basic question was asked recently: What is a better approximation to $e^x$, the usual Taylor approximation, or a similar approximation involving $1/e^{-x}$?

More precisely, given an integer $m$, which is a better approximation to $e^x$:

$$f_1(x) = \sum_{k=0}^m \frac{x^k}{k!} $$

or

$$f_2(x) = \frac1{\displaystyle \sum_{k=0}^m \frac{(-1)^k x^k}{k!}} $$

The answer is amazingly simple: if $m$ is even, then $f_2$ is more accurate, while if $m$ is odd, $f_1$ is more accurate. The proof below is, in my opinion, really nifty and worth showing here. Keep in mind that this result follows from the fact that we are dealing with a remarkable function with very special properties. Other functions will not exhibit such easily calculable results as shown below. (Anyone familiar with Pade approximates can attest to how sticky rational approximations to functions can get.)

Keep in mind that, by “accuracy,” I refer to truncation error rather than roundoff error. Truncation error is that which results from substituting a Taylor series with a polynomial of finite order. The accuracy, i.e., truncation error, of any Taylor expansion of a given order is given by the next order of the expansion. For example, consider the first order approximation to $1/e^{-x}$, $f_2$, and its second-order error:

$$\frac1{1-x} = 1+x+x^2+O(x^3)$$

This approximation has twice the error as the first order approximation to $e^x$, $1+x$, whose second order term is $x^2/2!$.

Now consider the second order approximations with third order errors:

$$\begin{align}\frac1{1-x+x^2/2} &= 1+\left (x-\frac{x^2}{2} \right ) + \left (x-\frac{x^2}{2} \right )^2+ \left (x-\frac{x^2}{2} \right )^3 + \cdots \\&= 1+x+\frac{x^2}{2} + O(x^4)\end{align}$$

Note that, for the $f_2$ approximation, the third order error vanishes! This is obviously not the case for $f_1$. Thus, for the second-order expansion, $f_2$ has a smaller truncation error than $f_1$. However, for the next order approximation, the expansion of $f_2$ is

$$1+x+\frac{x^2}{2}+\frac{x^3}{6}+\frac{x^4}{12} + O(x^5) $$

whereas the fourth order term in the direct, Taylor expansion of the exponential is $x^4/24$. Thus, as with the first order approximation, $f_2$ has twice the truncation error as $f_1$.

There is a systematic way to prove that the accuracy alternates with order oddness or evenness by considering the error in the expansion of $e^x \cdot e^{-x}=1$. The proof is actually not all that hard. Consider the finite approximations to the equation $e^x \cdot e^{-x}=1$:

$$\left (\sum_{k=0}^m \frac{x^k}{k!} \right ) \left (\sum_{k=0}^m (-1)^k \frac{x^k}{k!} \right ) $$

It may be easily shown that the coefficients of $x$, $x^2$, …, $x^m$ are zero. We now consider the first error term, i.e., the coefficient of $x^{m+1}$:

$$\frac{x^1}{1!} \frac{(-1)^m x^m}{m!} + \frac{x^2}{2!} \frac{(-1)^{m-1} x^{m-1}}{(m-1)!} +\cdots + \frac{x^m}{m!} \frac{(-1)^1 x^1}{1!} = \sum_{k=1}^m \frac{(-1)^{m-k+1}}{k! (m-k+1)!} x^{m+1}$$

Rewrite this sum, sans the $x^{m+1}$ term, as

$$\sum_{k=0}^{m-1} \frac{(-1)^{m-k}}{(k+1)! (m-k)!} = (-1)^m \sum_{k=0}^{m-1} \frac{(-1)^k}{k! (m-k)!} \frac1{k+1}$$

We can evaluate this sum by realizing that

$$(1-x)^m = \sum_{k=0}^m (-1)^k \binom{m}{k} x^k $$

so that

$$\sum_{k=0}^m (-1)^k \binom{m}{k} \frac1{k+1} = \int_0^1 dx \, (1-x)^m = \frac1{m+1}$$

Therefore, by subtracting off the last term in the sum, we find that

$$(-1)^m \sum_{k=0}^{m-1} \frac{(-1)^k}{k! (m-k)!} \frac1{k+1} = – \frac{1-(-1)^m}{(m+1)!}$$

Therefore, we now may say that

$$\frac1{\displaystyle \sum_{k=0}^m (-1)^k \frac{x^k}{k!} } =

\sum_{k=0}^m \frac{x^k}{k!} + \frac{1-(-1)^m}{(m+1)!} x^{m+1} +

O(x^{m+2})$$

For even values of $m$, the next term error is zero, which is smaller than that for the direct Taylor series, which has error $x^{m+1}/(m+1)!$. On the other hand, for odd values, the error is double that of the direct Taylor series. This agrees with the above examples.

This was indeed very surprising and beautiful (and at the same time not too difficult to understand).