Finite difference coefficient

In mathematics, to approximate a derivative to an arbitrary order of accuracy, it is possible to use the finite difference. A finite difference can be central, forward or backward.

This table contains the coefficients of the central differences, for several orders of accuracy and with uniform grid spacing:^[1]

For example, the third derivative with a second-order accuracy is

f ‴ ( x 0 ) ≈ − 1 2 f ( x − 2 ) + f ( x − 1 ) − f ( x + 1 ) + 1 2 f ( x + 2 ) h x 3 + O ( h x 2 ) , {\displaystyle f'''(x_{0})\approx {\frac {-{\frac {1}{2}}f(x_{-2})+f(x_{-1})-f(x_{+1})+{\frac {1}{2}}f(x_{+2})}{h_{x}^{3}}}+O\left(h_{x}^{2}\right),}

where h x {\displaystyle h_{x}} represents a uniform grid spacing between each finite difference interval, and x n = x 0 + n h x {\displaystyle x_{n}=x_{0}+nh_{x}} .

For the m {\displaystyle m} -th derivative with accuracy n {\displaystyle n} , there are 2 p + 1 = 2 ⌊ m + 1 2 ⌋ − 1 + n {\displaystyle 2p+1=2\left\lfloor {\frac {m+1}{2}}\right\rfloor -1+n} central coefficients a − p , a − p + 1 , . . . , a p − 1 , a p {\displaystyle a_{-p},a_{-p+1},...,a_{p-1},a_{p}} . These are given by the solution of the linear equation system

( 1 1 . . . 1 1 − p − p + 1 . . . p − 1 p ( − p ) 2 ( − p + 1 ) 2 . . . ( p − 1 ) 2 p 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ( − p ) 2 p ( − p + 1 ) 2 p . . . ( p − 1 ) 2 p p 2 p ) ( a − p a − p + 1 a − p + 2 . . . . . . . . . a p ) = ( 0 0 0 . . . m ! . . . 0 ) , {\displaystyle {\begin{pmatrix}1&1&...&1&1\\-p&-p+1&...&p-1&p\\(-p)^{2}&(-p+1)^{2}&...&(p-1)^{2}&p^{2}\\...&...&...&...&...\\...&...&...&...&...\\...&...&...&...&...\\(-p)^{2p}&(-p+1)^{2p}&...&(p-1)^{2p}&p^{2p}\end{pmatrix}}{\begin{pmatrix}a_{-p}\\a_{-p+1}\\a_{-p+2}\\...\\...\\...\\a_{p}\end{pmatrix}}={\begin{pmatrix}0\\0\\0\\...\\m!\\...\\0\end{pmatrix}},}

where the only non-zero value on the right hand side is in the ( m + 1 ) {\displaystyle (m+1)} -th row.

An open source implementation for calculating finite difference coefficients of arbitrary derivates and accuracy order in one dimension is available.^[2]
Given that the left-hand side matrix J T {\displaystyle \mathbf {J} ^{T}} is a transposed Vandermonde matrix, a rearrangement reveals that the coefficients are basically computed by fitting and deriving a 2 p {\displaystyle 2p} -th order polynomial to a window of 2 p + 1 {\displaystyle 2p+1} points. Consequently, the coefficients can also be computed as the m {\displaystyle m} -th order derivative of a fully determined Savitzky–Golay filter with polynomial degree 2 p {\displaystyle 2p} and a window size of 2 p + 1 {\displaystyle 2p+1} . For this, open source implementations are also available.^[3] There are two possible definitions which differ in the ordering of the coefficients: a filter for filtering via discrete convolution or via a matrix-vector-product. The coefficients given in the table above correspond to the latter definition.

The theory of Lagrange polynomials provides explicit formulas for the finite difference coefficients.^[4] For the first six derivatives we have the following:

Derivative	a 0 {\displaystyle a_{0}}	a p ( p ≠ 0 ) {\displaystyle a_{p}(p\neq 0)}
1	0 {\displaystyle 0}	1 ! ( − 1 ) p + 1 ( n ! ) 2 p ( n − p ) ! ( n + p ) ! {\displaystyle {\frac {1!(-1)^{p+1}(n!)^{2}}{p(n-p)!(n+p)!}}}
2	− 2 H n , 2 {\displaystyle -2H_{n,2}}	2 ! ( − 1 ) p + 1 ( n ! ) 2 p 2 ( n − p ) ! ( n + p ) ! {\displaystyle {\frac {2!(-1)^{p+1}(n!)^{2}}{p^{2}(n-p)!(n+p)!}}}
3	0 {\displaystyle 0}	3 ! ( − 1 ) p + 1 ( n ! ) 2 p 3 ( n − p ) ! ( n + p ) ! ( 1 − p 2 H n , 2 ) {\displaystyle {\frac {3!(-1)^{p+1}(n!)^{2}}{p^{3}(n-p)!(n+p)!}}(1-p^{2}H_{n,2})}
4	12 ( H n , 2 2 − H n , 4 ) {\displaystyle 12(H_{n,2}^{2}-H_{n,4})}	4 ! ( − 1 ) p + 1 ( n ! ) 2 p 4 ( n − p ) ! ( n + p ) ! ( 1 − p 2 H n , 2 ) {\displaystyle {\frac {4!(-1)^{p+1}(n!)^{2}}{p^{4}(n-p)!(n+p)!}}(1-p^{2}H_{n,2})}
5	0 {\displaystyle 0}	5 ! ( − 1 ) p + 1 ( n ! ) 2 p 5 ( n − p ) ! ( n + p ) ! ( 1 − p 2 H n , 2 + p 4 2 ( H n , 2 2 − H n , 4 ) ) {\displaystyle {\frac {5!(-1)^{p+1}(n!)^{2}}{p^{5}(n-p)!(n+p)!}}\left(1-p^{2}H_{n,2}+{\frac {p^{4}}{2}}(H_{n,2}^{2}-H_{n,4})\right)}
6	− 120 H n , 2 3 + 360 H n , 2 H n , 4 − 120 H n , 6 {\displaystyle -120H_{n,2}^{3}+360H_{n,2}H_{n,4}-120H_{n,6}}	6 ! ( − 1 ) p + 1 ( n ! ) 2 p 6 ( n − p ) ! ( n + p ) ! ( 1 − p 2 H n , 2 + p 4 2 ( H n , 2 2 − H n , 4 ) ) {\displaystyle {\frac {6!(-1)^{p+1}(n!)^{2}}{p^{6}(n-p)!(n+p)!}}\left(1-p^{2}H_{n,2}+{\frac {p^{4}}{2}}(H_{n,2}^{2}-H_{n,4})\right)}

where H n , m {\displaystyle H_{n,m}} are generalized harmonic numbers.

This table contains the coefficients of the forward differences, for several orders of accuracy and with uniform grid spacing:^[1]

For example, the first derivative with a third-order accuracy and the second derivative with a second-order accuracy are

f ′ ( x 0 ) ≈ − 11 6 f ( x 0 ) + 3 f ( x + 1 ) − 3 2 f ( x + 2 ) + 1 3 f ( x + 3 ) h x + O ( h x 3 ) , {\displaystyle \displaystyle f'(x_{0})\approx \displaystyle {\frac {-{\frac {11}{6}}f(x_{0})+3f(x_{+1})-{\frac {3}{2}}f(x_{+2})+{\frac {1}{3}}f(x_{+3})}{h_{x}}}+O\left(h_{x}^{3}\right),}

f ″ ( x 0 ) ≈ 2 f ( x 0 ) − 5 f ( x + 1 ) + 4 f ( x + 2 ) − f ( x + 3 ) h x 2 + O ( h x 2 ) , {\displaystyle \displaystyle f''(x_{0})\approx \displaystyle {\frac {2f(x_{0})-5f(x_{+1})+4f(x_{+2})-f(x_{+3})}{h_{x}^{2}}}+O\left(h_{x}^{2}\right),}

while the corresponding backward approximations are given by

f ′ ( x 0 ) ≈ 11 6 f ( x 0 ) − 3 f ( x − 1 ) + 3 2 f ( x − 2 ) − 1 3 f ( x − 3 ) h x + O ( h x 3 ) , {\displaystyle \displaystyle f'(x_{0})\approx \displaystyle {\frac {{\frac {11}{6}}f(x_{0})-3f(x_{-1})+{\frac {3}{2}}f(x_{-2})-{\frac {1}{3}}f(x_{-3})}{h_{x}}}+O\left(h_{x}^{3}\right),}

f ″ ( x 0 ) ≈ 2 f ( x 0 ) − 5 f ( x − 1 ) + 4 f ( x − 2 ) − f ( x − 3 ) h x 2 + O ( h x 2 ) , {\displaystyle \displaystyle f''(x_{0})\approx \displaystyle {\frac {2f(x_{0})-5f(x_{-1})+4f(x_{-2})-f(x_{-3})}{h_{x}^{2}}}+O\left(h_{x}^{2}\right),}

To get the coefficients of the backward approximations from those of the forward ones, give all odd derivatives listed in the table in the previous section the opposite sign, whereas for even derivatives the signs stay the same. The following table illustrates this:^[5]

For N {\displaystyle \displaystyle N} arbitrary stencil points s {\displaystyle \displaystyle s} and any derivative of order d < N {\displaystyle \displaystyle d<N} up to one less than the number of stencil points, the finite difference coefficients can be obtained by solving the linear equations ^[6]

( s 1 0 ⋯ s N 0 ⋮ ⋱ ⋮ s 1 N − 1 ⋯ s N N − 1 ) ( a 1 ⋮ a N ) = d ! ( δ 0 , d ⋮ δ i , d ⋮ δ N − 1 , d ) , {\displaystyle {\begin{pmatrix}s_{1}^{0}&\cdots &s_{N}^{0}\\\vdots &\ddots &\vdots \\s_{1}^{N-1}&\cdots &s_{N}^{N-1}\end{pmatrix}}{\begin{pmatrix}a_{1}\\\vdots \\a_{N}\end{pmatrix}}=d!{\begin{pmatrix}\delta _{0,d}\\\vdots \\\delta _{i,d}\\\vdots \\\delta _{N-1,d}\end{pmatrix}},}

where δ i , j {\displaystyle \delta _{i,j}} is the Kronecker delta, equal to one if i = j {\displaystyle i=j} , and zero otherwise.

Example, for s = [ − 3 , − 2 , − 1 , 0 , 1 ] {\displaystyle s=[-3,-2,-1,0,1]} , order of differentiation d = 4 {\displaystyle d=4} :

( a 1 a 2 a 3 a 4 a 5 ) = ( 1 1 1 1 1 − 3 − 2 − 1 0 1 9 4 1 0 1 − 27 − 8 − 1 0 1 81 16 1 0 1 ) − 1 ( 0 0 0 0 24 ) = ( 1 − 4 6 − 4 1 ) . {\displaystyle {\begin{pmatrix}a_{1}\\a_{2}\\a_{3}\\a_{4}\\a_{5}\end{pmatrix}}={\begin{pmatrix}1&1&1&1&1\\-3&-2&-1&0&1\\9&4&1&0&1\\-27&-8&-1&0&1\\81&16&1&0&1\\\end{pmatrix}}^{-1}{\begin{pmatrix}0\\0\\0\\0\\24\end{pmatrix}}={\begin{pmatrix}1\\-4\\6\\-4\\1\end{pmatrix}}.}

The order of accuracy of the approximation takes the usual form O ( h x ( N − d ) ) {\displaystyle O\left(h_{x}^{(N-d)}\right)} (or better in the case of central finite difference).^{[citation needed]}

• Finite difference method
• Finite difference
• Five-point stencil
• Numerical differentiation