Number - Fixed-point number

1 - About

A fixed-point number represents a limited-precision rational number that may have a fractional part.

These numbers are stored internally in a scaled-integer form, typically in binary but sometimes in decimal.

Because fixed-point numbers have limited precision, only a subset of real or rational numbers are exactly representable; other numbers can be represented only approximately.

Fixed-point numbers also tend to have a more limited range of values than floating point, and so the programmer must be careful to avoid overflow in intermediate calculations as well as the final results.

3 - Definition

3.1 - Precision

Precision is the total number of digits.

Example:

  • The number 00123456.78 has a precision of 8

If a value exceeds the precision, then Oracle returns an error.

3.2 - Scale

Scale is the number of digits from the radix point to the least significant digit.

Example:

  • The number 00123456.78 has a positive scale of 2

If a value exceeds the scale, then Oracle rounds it.

The scale can range from -84 to 127.

3.2.1 - Positive

Positive scale is the number of significant digits to the right of the decimal point to and including the least significant digit.

3.2.2 - Negative

Negative scale is the number of significant digits to the left of the decimal point, to but not including the least significant digit. For negative scale the least significant digit is on the left side of the decimal point, because the actual data is rounded to the specified number of places to the left of the decimal point. For example, a specification of (10,-2) means to round to hundreds.

4 - Documentation / Reference

data/type/number/fixed_point.txt · Last modified: 2017/09/13 16:04 by gerardnico