When deriving a finite difference approximation of the
th derivative of a function
, we wish to find
and
such that
![{\displaystyle f^{(j)}(x_{0})=h^{-j}\sum _{i=1}^{n}a_{i}f(x_{0}+b_{i}h)+O(h^{k}){\text{ as }}h\to 0}](https://wikimedia.org/api/rest_v1/media/math/render/svg/5fb5865282bd505483fbdd0a83836051e10f73a4)
or, equivalently,
![{\displaystyle h^{-j}\sum _{i=1}^{n}a_{i}f(x_{0}+b_{i}h)=f^{(j)}(x_{0})+O(h^{k}){\text{ as }}h\to 0}](https://wikimedia.org/api/rest_v1/media/math/render/svg/0b4f1d0ca24382c2e5ae012c151ec161c09c8269)
where
is the error, the difference between the correct answer and the approximation, expressed using Big-O notation. Because
may be presumed to be small, a larger value for
is better than a smaller value.
A general method for finding the coefficients is to generate the Taylor expansion of
and choose
and
such that
and the remainder term are the only non-zero terms. If there are no such coefficients, a smaller value for
must be chosen.
For a function of
variables
, the procedure is similar, except
are replaced by points in
and the multivariate extension of Taylor's theorem is used.
In all single-variable examples,
and
are unknown, and
is small. Additionally, let
be 5 times continuously differentiable on
.
Find
such that
best approximates
.
Let
be 42 times continuously differentiable on
. Find the largest
such that
![{\displaystyle {\frac {df}{dx}}(x_{0})={\frac {-f(x_{0}+2h)+8f(x_{0}+h,y_{0})-8f(x_{0}-h)+f(x_{0}-2h)}{12h}}+O(h^{n}){\text{ as }}h\to 0}](https://wikimedia.org/api/rest_v1/media/math/render/svg/0eb8529a4bdd0c6d6ab8128bd66a2d0640e69194)
In other words, find the order of the error of the method.
Solution:
The Taylor expansion of the method is
![{\displaystyle {\begin{aligned}{\frac {-f(x_{0}+2h)+8f(x_{0}+h,y_{0})-8f(x_{0}-h)+f(x_{0}-2h)}{12h}}&={\frac {-1+8-8+1}{12h}}f(x_{0})+{\frac {-2+8+8-2}{12}}f'(x_{0})\\&+h{\frac {(-4+8-8+4)}{24}}f''(x_{0})+h^{2}{\frac {(-8+8+8-8}{72}}f'''(x_{0})\\&+h^{3}{\frac {(-16+8-8+16}{288}}f^{(4)}(x_{0})+h^{4}{\frac {(-32+8+8-32)}{1440}}f^{(4)}(x_{0})+O(h^{5}){\text{ as }}h\to 0\,.\end{aligned}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/c6736569932b2507610c673a4f319327b853102b)
Simplifying this algebraically gives
![{\displaystyle {\frac {-f(x_{0}+2h)+8f(x_{0}+h,y_{0})-8f(x_{0}-h)+f(x_{0}-2h)}{12h}}=f'(x_{0})+{\frac {h^{4}}{-30}}f^{(5)}(x_{0})+O(h^{5}){\text{ as }}h\to 0\,.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/0f2a8d68d9f46399a49b49588f55fa1d3ec509de)
The multiple of
cannot be removed, so
and by properties of Big-O notation,
![{\displaystyle {\frac {-f(x_{0}+2h)+8f(x_{0}+h,y_{0})-8f(x_{0}-h)+f(x_{0}-2h)}{12h}}=f'(x_{0})+O(h^{4}){\text{ as }}h\to 0\,.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/9cd75e4051e71fdff2fe261ac5edb083e67a7836)
Find
such that
best approximates
.
In all two-variable examples,
and
are unknown, and
is small.
Because of the nature of partial derivatives, some of them may be calculated using single-variable methods. This is done by holding constant all but one variable to form a new function of one variable. For example if
, then
.
Find an approximation of
Solution:
Because we are differentiating with respect to only one variable, we can hold x constant and use the result of one of the single-variable examples:
![{\displaystyle {\frac {f(x_{0},y_{0}+h)-f(x_{0},y_{0}-h)}{2h}}={\frac {df}{dy}}(x_{0},y_{0})+O(h^{2}){\text{ as }}h\to 0}](https://wikimedia.org/api/rest_v1/media/math/render/svg/1d65a76a1d0739d5caa447b91082fc9258134d8d)
Mixed derivatives may require the multivariate extension of Taylor's theorem.
Let
be 42 times continuously differentiable on
and let
be defined as
![{\displaystyle g(x_{0},y_{0},h)={\frac {f(x_{0}+h,y_{0}+h)+f(x_{0}-h,y_{0}-h)-f(x_{0}-h,y_{0}+h)-f(x_{0}+h,y_{0}-h)}{4h^{2}}}\,.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/540b63372b25b4988d0222a40d5870f23ada0396)
Find the largest
such that
![{\displaystyle {\frac {d^{2}f}{dxdy}}(x_{0},y_{0})=g(x_{0},y_{0},h)+O(h^{n}){\text{ as }}h\to 0\,.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/f31b560058b51135c46c0b4272b96ee430c584a2)
In other words, find the order of the error of the approximation.
Solution:
The first few terms of the multivariate Taylor expansion of
around
are
![{\displaystyle {\begin{aligned}f(x_{0}+x,y_{0}+y)&=f(x_{0},y_{0})\\&+x{\frac {df}{dx}}(x_{0},y_{0})+y{\frac {df}{dy}}(x_{0},y_{0})\\&+{\frac {x^{2}}{2}}{\frac {d^{2}f}{dx^{2}}}(x_{0},y_{0})+xy{\frac {d}{dy}}{\frac {df}{dx}}(x_{0},y_{0})+{\frac {y^{2}}{2}}{\frac {d^{2}f}{dy^{2}}}(x_{0},y_{0})\\&+{\frac {x^{3}}{6}}{\frac {d^{3}f}{dx^{3}}}(x_{0},y_{0})+{\frac {x^{2}y}{2}}{\frac {d}{dy}}{\frac {d^{2}f}{dx^{2}}}(x_{0},y_{0})+{\frac {xy^{2}}{2}}{\frac {d^{2}}{dy^{2}}}{\frac {df}{dx}}(x_{0},y_{0})+{\frac {y^{3}}{6}}{\frac {d^{3}f}{dy^{3}}}(x_{0},y_{0})\\&+{\frac {x^{4}}{24}}{\frac {d^{4}f}{dx^{4}}}(x_{0},y_{0})+{\frac {x^{3}y}{6}}{\frac {d}{dy}}{\frac {d^{3}f}{dx^{3}}}(x_{0},y_{0})+{\frac {x^{2}y^{2}}{4}}{\frac {d^{2}}{dy^{2}}}{\frac {d^{2}f}{dx^{2}}}(x_{0},y_{0})+{\frac {xy^{3}}{6}}{\frac {d^{3}}{dy^{3}}}{\frac {df}{dx}}(x_{0},y_{0})+{\frac {y^{4}}{24}}{\frac {d^{4}f}{dy^{4}}}(x_{0},y_{0})\\&+O(x^{5})+O(y^{5}){\text{ as }}x,y\to 0\,.\end{aligned}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/496bb9f56830fd3143a86aa6ba074a9d75436f62)
We substitute the expansion for
into the approximation
to obtain
![{\displaystyle {\begin{aligned}g(x_{0},y_{0},h)&={\frac {1+1-1-1}{4h^{2}}}f(x_{0},y_{0})\\&+{\frac {1-1+1-1}{4h}}{\frac {df}{dx}}(x_{0},y_{0})+{\frac {1-1-1+1}{4h}}{\frac {df}{dy}}(x_{0},y_{0})\\&+{\frac {1+1-1-1}{8}}{\frac {d^{2}f}{dx^{2}}}(x_{0},y_{0})+{\frac {1+1+1+1}{4}}{\frac {d}{dy}}{\frac {df}{dx}}(x_{0},y_{0})+{\frac {1+1-1-1}{8}}{\frac {d^{2}f}{dy^{2}}}(x_{0},y_{0})\\&+h\left({\frac {1-1+1-1}{24}}{\frac {d^{3}f}{dx^{3}}}(x_{0},y_{0})+{\frac {1-1-1+1}{8}}{\frac {d}{dy}}{\frac {d^{2}f}{dx^{2}}}(x_{0},y_{0})+{\frac {1-1+1-1}{8}}{\frac {d^{2}}{dy^{2}}}{\frac {df}{dx}}(x_{0},y_{0})+{\frac {1-1-1+1}{24}}{\frac {d^{3}f}{dy^{3}}}(x_{0},y_{0})\right)\\&+h^{2}\left({\frac {1+1-1-1}{96}}{\frac {d^{4}f}{dx^{4}}}(x_{0},y_{0})+{\frac {1+1+1+1}{24}}{\frac {d}{dy}}{\frac {d^{3}f}{dx^{3}}}(x_{0},y_{0})+{\frac {1+1-1-1}{16}}{\frac {d^{2}}{dy^{2}}}{\frac {d^{2}f}{dx^{2}}}(x_{0},y_{0})+{\frac {1+1+1+1}{24}}{\frac {d^{3}}{dy^{3}}}{\frac {df}{dx}}(x_{0},y_{0})+{\frac {1+1-1-1}{96}}{\frac {d^{4}f}{dy^{4}}}(x_{0},y_{0})\right)\\&+O(h^{3}){\text{ as }}h\to 0\,.\end{aligned}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/44d63472ec7003e351e48e4dce97fd6fe34a75a8)
Because of the careful choices of coefficients, we can simplify this to
![{\displaystyle g(x_{0},y_{0},h)={\frac {d}{dy}}{\frac {df}{dx}}(x_{0},y_{0})+{\frac {h^{2}}{6}}\left({\frac {d}{dy}}{\frac {d^{3}f}{dx^{3}}}(x_{0},y_{0})+{\frac {d^{3}}{dy^{3}}}{\frac {df}{dx}}(x_{0},y_{0})\right)+O(h^{3}){\text{ as }}h\to 0\,.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/e75a9d2c38d77520318cf5c5fb39b16309f36ca1)
We note that Big-O notation permits us to write the last 3 terms as
. Thus,
![{\displaystyle g(x_{0},y_{0},h)={\frac {d}{dy}}{\frac {df}{dx}}(x_{0},y_{0})+O(h^{2}){\text{ as }}h\to 0\,.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/243ef6b9094e410eab6c8d06cc7a791683a7ea99)
Because the multiples of
are unaffected by adding more terms to the Taylor expansion,
is the greatest natural number satisfying the conditions given in the problem.
Implementing these methods is reasonably simple in programming languages that support higher-order functions. For example, the method from the first example may be implemented in C++ using function pointers, as follows:
// Returns an approximation of the derivative of f at x.
double derivative (double (*f)(double), double x, double h =0.01) {
return (f(x + h) - f(x - h)) / (2 * h);
}