Numerical error

Numerical error is the error that arises in numerical computations due to limitations in representing real numbers and performing arithmetic operation on computers. These errors commonly appear in software engineering and mathematics.

Types

Numerical error in computations generally arises from several sources. The most common sources are round-off error and truncation error.^[1]

It can be the combined effect of two kinds of error in a calculation. The first is referred to as round-off error and is caused by the finite precision of computations involving floating-point numbers. The second, usually called truncation error, is the difference between the exact mathematical solution and the approximate solution obtained when simplifications are made to the mathematical equations to make them more amenable to calculation.

Measure

Floating-point numerical error is often measured in ULP (unit in the last place), which represents the distance between two adjacent floating-point numbers.^[2]

References

^ Burden, Richard; Faires, Douglas (August 9, 2010). Numerical Analysis (9th ed.). Cengage Learning. ISBN 978-0538733519.
^ "Units in the last pace (ULP)". Arm Learning Paths. Retrieved March 11, 2026.

Accuracy and Stability of Numerical Algorithms, Nicholas J. Higham, ISBN 0-89871-355-2
"Computational Error And Complexity In Science And Engineering", V. Lakshmikantham, S.K. Sen, ISBN 0444518606

This software-engineering-related article is a stub. You can help Wikipedia by adding missing information.

This applied mathematics–related article is a stub. You can help Wikipedia by adding missing information.

[1] Burden, Richard; Faires, Douglas (August 9, 2010). Numerical Analysis (9th ed.). Cengage Learning. ISBN 978-0538733519.

[2] "Units in the last pace (ULP)". Arm Learning Paths. Retrieved March 11, 2026.

[1]

[2]

Numerical error

Types

Measure

See also

References