The inner workings of exponent bias in floating point
In most cases, signed integers are represented using two’s complement. However, in the IEEE-754 floating point standard, the exponent is stored as offset binary. This approach is also referred to as a biased exponent or offset-k, where k stands for the offset value. When researching how to encode a number with this scheme, you’ll typically read that you compute a bias, then add it to the target number. That sum is what gets stored. If needed, the result is then converted to binary.
To illustrate the process, consider how the number 3 would be stored in 4 bits:
- Determine the bias using the formula from the IEEE-754 standard:
Here, n represents the bit count. For 4 bits, the bias comes out to 7.
2. Add the bias to the original value: 3 + 7 = 10. The resulting value, 10, is what represents the number 3 in the offset binary scheme. Since the result 10 was obtained in decimal, it must be converted to binary:
If the conversion process is unfamiliar, or you’re curious about why it works, feel free to check out my piece on decimal-binary conversion algorithms.
Understanding the bias
Let’s say we only have 4 bits available for storing numbers. How many distinct values can be represented with 4 bits? The answer is straightforward. We apply the formula for permutations with repetition:
In this formula, n indicates the number of choices, and we select r of them. So, we have 2 (n) possible symbols — 1 and 0 — and we aim to pick 4 (r) of them, as we have 4 bits:
This computation reveals that 16 unique numbers can be stored in 4 bits. The question becomes: what exactly are these numbers? If our interest is limited to non-negative integers, the range is:
But when negative integers are included, the range can shift:
An intriguing observation is that while the decimal range changes, the binary range stays uniform. The only difference is that the lowest binary value, 0000, now maps to a negative number. It’s as if this minimum value is pushed downward from zero by 1 in the first scenario, 7 in the second, and 8 in the third.
I’ve presented three different ranges, and it might be unclear which one represents the correct distribution. As it turns out, no single standard dictates how numbers should be split across the negative and non-negative sides of zero. A common practice is to balance the count of numbers on each side. For 4 bits, this yields a range of [-8;7], with 8 negative numbers [-8;-1] and an equal count of non-negative numbers [0;7]. Yet, the IEEE-754 standard opts for a slightly different layout, giving the non-negative side two extra numbers compared to the negative side — resulting in [-7;8].
To compute the representation of any number under the offset binary system, we need the method for determining the bias. Let’s start with the balanced case, where negative and non-negative counts are equal. If there are 2⁴ numbers in total, half amounts to 2³, so the formula for this bias is:
In this context, n is the bit count. A bias of 8 means we shift the range down by 8 numbers from zero [-8;7], making -8 the lowest representable value.
As noted earlier, IEEE-754 chose to allocate more room for positive values. Consequently, the negative range is reduced by one number, leading to this bias formula:
Using this formula, the bias for 4 bits is 7, which yields the range [-7; 8].
Encoding values
With the bias calculation understood, let’s look at how it’s used. As mentioned at the start, when converting a number to offset binary, the bias is added to the original value, and to retrieve the original, it’s subtracted. There’s simple arithmetic behind this.
Take, for example, storing the number 3 in 4 bits. We pick the bias 7, derived from the IEEE-754 formula. If 0000 represents -7, what value must we add to reach 3? That would be 10. Let’s see the result:
This demonstrates that 3 is encoded as 1010 in binary with a bias of 7. It should also be clear where the operation of adding the bias to derive the offset binary representation comes from:
Consequently, since we add the bias to encode the number in offset binary, we subtract it to decode the number back to its original form.
Why it beats two’s complement
A key benefit of offset binary compared to two’s complement is that numbers can be compared directly using lexicographic order without extra steps. For instance, let’s compare the numbers 3 and -3 using 4-bit representation. In offset binary, they look like this:
When comparing bit by bit, the computer can see immediately from the leading bit that the first value is larger. Lexicographic order can’t be applied to two’s complement numbers:
To compare these, the computer must carry out extra operations.