# Numerical Analysis/Differentiation/Examples

When deriving a finite difference approximation of the $j$ th derivative of a function $f:\mathbb {R} \rightarrow \mathbb {R}$ , we wish to find $a_{1},a_{2},...,a_{n}\in \mathbb {R}$ and $b_{1},b_{2},...,b_{n}\in \mathbb {R}$ such that

$f^{(j)}(x_{0})=h^{-j}\sum _{i=1}^{n}a_{i}f(x_{0}+b_{i}h)+O(h^{k}){\text{ as }}h\to 0$ or, equivalently,

$h^{-j}\sum _{i=1}^{n}a_{i}f(x_{0}+b_{i}h)=f^{(j)}(x_{0})+O(h^{k}){\text{ as }}h\to 0$ where $O(h^{k})$ is the error, the difference between the correct answer and the approximation, expressed using Big-O notation. Because $h$ may be presumed to be small, a larger value for $k$ is better than a smaller value.

A general method for finding the coefficients is to generate the Taylor expansion of $h^{-j}\sum _{i=1}^{n}a_{i}f(x_{0}+b_{i}h)$ and choose $a_{1},a_{2},...,a_{n}$ and $b_{1},b_{2},...,b_{n}$ such that $f^{(j)}(x_{0})$ and the remainder term are the only non-zero terms. If there are no such coefficients, a smaller value for $k$ must be chosen.

For a function of $m$ variables $g:\mathbb {R} ^{m}\rightarrow \mathbb {R}$ , the procedure is similar, except $x_{0},b_{1},b_{2},...,b_{n}$ are replaced by points in $\mathbb {R} ^{m}$ and the multivariate extension of Taylor's theorem is used.

## Single-Variable

In all single-variable examples, $x_{0}\in \mathbb {R}$ and $f:\mathbb {R} \rightarrow \mathbb {R}$ are unknown, and $h\in \mathbb {R}$ is small. Additionally, let $f$ be 5 times continuously differentiable on $\mathbb {R}$ .

### First Derivative

Find $a,b,c\in \mathbb {R}$ such that ${\frac {af(x_{0}+h)+bf(x_{0}+ch)}{h}}$ best approximates $f'(x_{0})$ .

Let $f:\mathbb {R} \rightarrow \mathbb {R}$ be 42 times continuously differentiable on $\mathbb {R}$ . Find the largest $n\in \mathbb {N}$ such that

${\frac {df}{dx}}(x_{0})={\frac {-f(x_{0}+2h)+8f(x_{0}+h,y_{0})-8f(x_{0}-h)+f(x_{0}-2h)}{12h}}+O(h^{n}){\text{ as }}h\to 0$ In other words, find the order of the error of the method.

### Second Derivative

Find $a,b,c\in \mathbb {R}$ such that ${\frac {af(x_{0}-h)+bf(x_{0})+cf(x_{0}+h)}{h^{2}}}$ best approximates $f''(x_{0})$ .

## Multivariate

In all two-variable examples, $x_{0},y_{0}\in \mathbb {R}$ and $f:\mathbb {R} ^{2}\rightarrow \mathbb {R}$ are unknown, and $h\in \mathbb {R}$ is small.

### Non-Mixed Derivatives

Because of the nature of partial derivatives, some of them may be calculated using single-variable methods. This is done by holding constant all but one variable to form a new function of one variable. For example if $g_{y}(y)=f(x_{0},y)$ , then ${\frac {df}{dy}}(x_{0},y_{0})={\frac {dg}{dy}}(y_{0})$ .

Find an approximation of ${\frac {d}{dy}}f(x_{0},y_{0})$ ### Mixed Derivatives

Mixed derivatives may require the multivariate extension of Taylor's theorem.

Let $f:\mathbb {R} ^{2}\rightarrow \mathbb {R}$ be 42 times continuously differentiable on $\mathbb {R} ^{2}$ and let $g:\mathbb {R} ^{3}\rightarrow \mathbb {R}$ be defined as

$g(x_{0},y_{0},h)={\frac {f(x_{0}+h,y_{0}+h)+f(x_{0}-h,y_{0}-h)-f(x_{0}-h,y_{0}+h)-f(x_{0}+h,y_{0}-h)}{4h^{2}}}\,.$ Find the largest $n\in \mathbb {N}$ such that

${\frac {d^{2}f}{dxdy}}(x_{0},y_{0})=g(x_{0},y_{0},h)+O(h^{n}){\text{ as }}h\to 0\,.$ In other words, find the order of the error of the approximation.

## Example Code

Implementing these methods is reasonably simple in programming languages that support higher-order functions. For example, the method from the first example may be implemented in C++ using function pointers, as follows:

//  Returns an approximation of the derivative of f at x.
double derivative (double (*f)(double), double x, double h =0.01) {
return (f(x + h) - f(x - h)) / (2 * h);
}