Bits, precision, and rounding - Inside the Machine: The Mathematics of Small Language Models

Chapter 6. Bits, precision, and rounding¶

Where a model’s gigabytes come from, and what a number loses when you store it in fewer bits.

What you need before this chapter¶

This chapter is arithmetic. Multiplication, division, and one kind of exponent. There is no calculus in it, and no algebra beyond a letter standing for a number.

Here is the honest list of what it uses. Every item links to a page in the Math Toolkit that starts from nothing and builds up. Open a link the moment you want it. You are not expected to arrive knowing any of this.

What this chapter uses	Where it is taught from zero
A letter standing for a number, such as $b$ for “how many bits”	Toolkit 1, A letter standing for a number
Exponents, such as 2⁴	Toolkit 5, Exponents
Subscripts, the small labels in $g_{b}$ and $g_{b-1}$ , in Section 6.5 only	Toolkit 2, Subscripts
Powers of two, and what a bit is	Toolkit 18, Powers of two, and what a bit is
Multiplication, written four different ways	Toolkit 3, Multiplication
The fraction bar and the division sign $\div$ , which both mean divide	Toolkit 4, The fraction bar is division
Proportions, and what “twice as wide” does to an answer, in Section 6.5 only	Toolkit 12, Proportions
Rounding, and decimal places	Toolkit 16, Rounding
Absolute value, which is the size of a number with the minus sign thrown away	Toolkit 15, Absolute value and magnitude
Scientific notation, because 494,032,768 is tiring to write out	Toolkit 17, Scientific notation
A logarithm, in one place only, in Section 6.1	Toolkit 8, Logarithms
Percentages, and how a decimal such as 0.125 becomes 12.5%	Toolkit 11, Percentages
Standard deviation, mentioned once in Section 6.3 and never computed here	Toolkit 10, Formula 9

One idea carries over from earlier in the book, and here it is again in full, so that you do not have to go and find it.

A parameter is one number that a model learned during training and now keeps. It is also called a weight, and this book uses the two words for the same thing. A model is a very large pile of these numbers, arranged into tables, together with the instructions for what to multiply and add. The smallest model in this course, Qwen2.5-0.5B-Instruct, holds 494,032,768 parameters. That count was produced by lab/we3_params_quant.py and is recorded in lab/out/we3_params_quant.json.

Every parameter has to be written down somewhere. This chapter is about what “written down” costs.

Setup¶

Run this cell once, at the top of your session. Every import this chapter needs is in it, one per line, each with a comment saying what it is for. This chapter draws no charts, so it does not import a charting library.

# Cell 1. Run this once, at the top of the session, before any other cell in this chapter.

import json    # reads .json files, which is how every lab in this book saves its numbers
import math    # logarithms; this chapter uses one logarithm, in Section 6.1, and nothing else

# Every file path below is written from the top folder of the course repository, the folder
# that holds both "book" and "lab". Start your notebook there and the paths will resolve.
path_to_the_measured_file = "lab/out/we3_params_quant.json"   # the real run this chapter checks itself against

There is no model to load in this chapter and no graphics card involved. Everything here is multiplication and division applied to numbers that an earlier run already measured and saved to a file you can open.

The number on the download bar¶

The first thing this course asked you to do was download a model. While the file came down, your browser showed you a number. On the machine this book was written on, that number was 0.99 GB.

Nobody chose 0.99. It is not a marketing size. It is not a round figure somebody picked in a meeting. It is the answer to a single multiplication, and by the end of this chapter you will be able to do that multiplication yourself, for any model you meet, before you download it.

Here is the multiplication in advance, so you know where the chapter is heading. Qwen2.5-0.5B-Instruct holds 494,032,768 parameters. Each parameter is written down using 16 bits, and 16 bits is 2 bytes. Multiply: $494{,}032{,}768 \times 2 = 988{,}065{,}536$ bytes, which is 0.988 gigabytes. The download bar rounded it to 0.99 and moved on. That is the whole mystery of model file sizes, and the arithmetic is identical for a model a thousand times larger.

Now the second half of the chapter, and it starts at a fruit stand.

Drive north out of Bakersfield on Highway 99 in July and you pass stands selling table grapes by the pound. The scale on the counter has a display with two digits after the point. Put three and a bit pounds of grapes on it and it reads 3.14. It does not read 3.14159. The scale is not broken. The grapes did not change weight when you looked at the display. The display has a fixed number of digits, so there is a fixed list of readings it is able to show, and the true weight gets pushed to the nearest reading on that list.

Every number inside a language model lives on a list like that. The list is shorter than you might expect. The gaps between the entries on the list decide how much detail survives. And the length of the list is set by one choice: how many bits you are willing to spend. Fewer bits means a smaller file and a shorter list. A shorter list means bigger gaps. Bigger gaps mean more of the original number gets pushed away.

That trade is the subject of this chapter and of the whole of Module C. This chapter builds the arithmetic. Chapter 7 spends it.

Where every number in this chapter comes from

This book follows one rule with no exceptions.

We never publish a number we did not compute.

Every measured number in this chapter traces back to a script in lab/ and a JSON file in lab/out/, and the text names the file at the point the number appears. The parameter counts and the real weight matrix come from lab/out/we3_params_quant.json, produced by lab/we3_params_quant.py running on the real Qwen2.5-0.5B-Instruct weights. The file sizes in half precision come from lab/out/lab4_size_ladder.json. The scales and the rounding errors come from lab/out/we3_params_quant.json and lab/out/we3b_quant.json.

Some examples in this chapter use small invented numbers, because 3 and 10 are easier to check on a phone than 0.02697754 is. Every one of those is labelled made up for practice at the point where it appears. Anything not labelled that way is a real measurement.

If you find a number in this chapter that you cannot trace back to a file, that is a bug, and your instructor wants to hear about it.

Learning objectives¶

By the end of this chapter you will be able to:

Say what a bit is in one sentence, and work out how many different values $b$ bits can hold.
Turn a bit width into bytes, and turn a parameter count into a file size in gigabytes, for any model, without downloading it.
Draw the levels that a storage format is able to write down, and compute the gap between two neighbouring levels.
Round a number to the nearest level, and state the largest error that rounding can possibly cause.
Explain why the range a format has to cover matters as much as the number of bits it spends, which is the idea the whole of Chapter 7 is built on.

This lesson at a glance¶

A bit is one yes-or-no answer. With $b$ bits you can name $2^{b}$ different values and nothing in between them.
A model’s file size is a multiplication: the parameter count times the bytes spent on each parameter. 494,032,768 parameters at 2 bytes each is 0.988 GB.
The values a format can write down sit on a ruler with an even gap between them, and that gap is what the word “precision” actually means.
Rounding a number to the nearest level is wrong by at most half a gap. The largest errors the lab measured on a real weight matrix agree with half a gap to seven decimal places.

The vocabulary of this chapter¶

Every term this chapter uses is listed here, before it is used. Read the table once now. You do not need to memorise it. Each term is defined again, at more length, at the point where it starts doing work.

Term	In one line
bit	One yes-or-no answer, written 0 or 1. The smallest piece of information there is.
bit width	How many bits you spend on one number. Written $b$ in the formulas below.
byte	Eight bits, taken as a group. File sizes are counted in bytes.
gigabyte (GB)	One thousand million bytes, that is $1{,}000{,}000{,}000$ bytes.
parameter, or weight	One number the model learned during training and now keeps.
level	One of the values a format is able to write down exactly. Also called a representable value.
gap, or step	The distance from one level to the next one along. Written $g$ below.
range	How far the levels have to stretch, from the most negative one to the most positive one.
magnitude	The size of a number once you throw away its minus sign. The magnitude of -3 is 3.
rounding	Replacing a number with the nearest level.
rounding error	How far the number moved when you rounded it.
precision	How finely a format can tell nearby numbers apart. Small gaps mean high precision.
factor	What you multiply by. “A factor of 18” means “18 times as large”.
spread	The standard deviation of a group of numbers: one number saying how far a typical member sits from their average. Used only when quoting Chapter 7.
per-tensor	One ruler for a whole weight matrix, which is the scheme this chapter builds.
blockwise	One ruler for each small group of weights instead. Worked example 6.5 prices it; Chapter 7 does it.
floating point	The family of formats a computer uses for numbers that are not whole.
FP32	A floating point format that spends 32 bits, that is 4 bytes, on each number. Full precision.
FP16	A floating point format that spends 16 bits, that is 2 bytes, on each number. Half precision.
INT8	A format that spends 8 bits, that is 1 byte, on each number, and stores whole numbers.
quantization	Deliberately storing weights with fewer bits, and accepting the rounding that follows. The subject of Chapter 7.

6.1 What a bit is, and how many values bits can hold¶

Intuition¶

Picture a row of light switches on a classroom wall. Each switch is either up or down. There is no third position and no halfway. That is the whole idea of a bit: one thing that has exactly two possible states. Write the two states as 1 and 0 and you have the notation computers use.

One switch gives you two possible arrangements of the wall: up, or down.

Two switches give you four. The first switch can be up or down, and for each of those the second switch can also be up or down. Up-up, up-down, down-up, down-down. Four.

Three switches give you eight. Every one of the four arrangements you already had can now be followed by a third switch being up, or by it being down, so the count doubles again. Four becomes eight.

That doubling never stops and it never slows down. Each switch you add multiplies the number of arrangements by two, because every arrangement you already had splits into two.

Here is the consequence that matters for the rest of this book. A fixed number of switches gives you a fixed-size menu of names. Four switches can name 16 different things. Not 17. If you have 17 things to name, four switches are not enough, and you have to add a fifth, which takes you straight to 32 names with 15 of them spare. There is no way to buy half a switch.

This is why computers are strange about numbers. The number line between 0 and 1 has no gaps in it. You can always find a number between any two numbers, forever. A wall of switches is the opposite. It has a finite list of arrangements and nothing at all in between them. So storing a number in a computer is never “writing the number down”. It is picking the closest item on a menu.

The rest of this chapter is about how long that menu is, how far apart the items on it sit, and how much a number moves when it gets pushed to the nearest one.

The mathematics¶

Here is the first formula of the chapter. It answers one question: how many different values can you name with a given number of bits?

Formula 6.1, how many values fit in

b

bits

1. In words. The number of different values you can write down is two, multiplied by itself once for every bit you have.

2. The formula.

L = 2^{b}

3. The symbols.

Symbol	How to say it out loud	What it means
$L$	“capital ell”	the answer: how many different values you can name. $L$ is for levels.
$=$	“equals”	the thing on the left is the same number as the thing on the right
2	“two”	the base. It is a 2 because one bit has exactly two states, 0 and 1.
$b$	“bee”	the bit width: how many bits you are spending. A whole number, such as 4 or 16.
$2^{b}$	“two to the b”, or “two to the power of b”	2 multiplied by itself $b$ times. See Toolkit 5.
the raised $b$	“to the power of”	the small raised number tells you how many copies of the 2 to multiply together. It is not a multiplication by $b$ .
$\times$	“times”	multiply. It is used in the worked example below to write the doubling out in full. See Toolkit 3.

4. Out loud. “The number of levels is two to the power of the number of bits.”

5. Worked, with numbers made up for practice. How many different values fit in 4 bits?

Step 1, write out what 2⁴ means. The raised 4 says “use four copies of the 2”.

2^{4} = 2 \times 2 \times 2 \times 2

Step 2, multiply them one pair at a time, left to right. Do not try to do it in one go.

$2 \times 2 = 4$

$4 \times 2 = 8$

$8 \times 2 = 16$

Step 3, write the answer.

2^{4} = 16

Four bits hold 16 different values.

Worked again, at 8 bits. Carry on doubling from where you stopped.

$2^{4} = 16$

$2^{5} = 16 \times 2 = 32$

$2^{6} = 32 \times 2 = 64$

$2^{7} = 64 \times 2 = 128$

$2^{8} = 128 \times 2 = 256$

Eight bits hold 256 different values.

6. Check it. Every extra bit must exactly double the answer. Read your results in order and check that each one is twice the one before: 16, 32, 64, 128, 256. Each is double its predecessor, so the arithmetic is sound. If your answer for 5 bits is 20, you multiplied by the bit width instead of doubling, which is the most common slip here. And the answer can never be odd, except for $2^{0} = 1$ , because everything after the first step is a doubling.

This table is worth keeping in view for the rest of Module C. Every entry in it is $2^{b}$ worked out.

Bit width $b$	Levels $L = 2^{b}$	Where it turns up in this course
1	2	one yes-or-no
2	4	the harshest setting Chapter 7 tries
3	8
4	16	4-bit quantization, the setting that makes big models fit small machines
6	64
8	256	one byte, and also 8-bit quantization
16	65,536	FP16, the format every model in this course is stored in
32	4,294,967,296	FP32, full precision

Now turn the question around. Instead of “how many values do I get from $b$ bits”, ask “how many bits do I need to name $L$ things”. The tool for reversing an exponent is a logarithm, and it is covered from scratch in Toolkit 8. A logarithm to base 2, written $\log_2$ , tells you the exponent: $\log_2(16) = 4$ , because 4 is the power you have to raise 2 to in order to land on 16.

Here is why that matters. The model in this course has a vocabulary of 151,936 tokens, which is the number of distinct pieces of text it can choose between. That count is real, from lab/out/we2_softmax.json. Every one of those tokens needs its own number, so how many bits does a token id need? Seventeen bits give $2^{17} = 131{,}072$ names, which is short. Eighteen bits give $2^{18} = 262{,}144$ names, which is enough with room to spare. So a token id needs 18 bits, and $\log_2(151{,}936) = 17.213$ , rounded to three decimal places, tells you the same thing in one step: you need more than 17, therefore you need 18. You cannot buy 0.213 of a bit.

Try it 6.1

How many different values fit in 5 bits? In 10 bits?
A course needs to give every one of 500 students a different id number made of bits. How many bits does it need? These student numbers are made up for practice.
Somebody tells you that going from 4 bits to 8 bits doubles the number of levels. Are they right?

Python¶

The two cells below do the arithmetic you did by hand. Run them and compare.

The first one loops over a list of bit widths and prints the level count for each. Eric Van Dusen’s teaching style, which this book follows, asks for an explicit for loop rather than anything shorter, and for variable names that say what they hold. So the loop variable is called bit_width, not b.

# Cell 2. How many values fit in a given number of bits.

# The bit widths this course actually meets, smallest first.
bit_widths_to_show = [1, 2, 3, 4, 6, 8, 16, 32]

# Walk the list one entry at a time. On each pass, bit_width holds the next number in the list.
for bit_width in bit_widths_to_show:
    number_of_levels = 2 ** bit_width          # ** is how Python writes "to the power of"
    print("bits:", bit_width, " levels:", format(number_of_levels, ","))

bits: 1  levels: 2
bits: 2  levels: 4
bits: 3  levels: 8
bits: 4  levels: 16
bits: 6  levels: 64
bits: 8  levels: 256
bits: 16  levels: 65,536
bits: 32  levels: 4,294,967,296

Three things in that output are worth stopping on.

The line for 4 bits says 16, which is the number you worked out by hand. The line for 8 bits says 256, which is the other one you worked out. So the machine and the paper agree.

The jump from 16 bits to 32 bits is worth reading slowly. Doubling the bit width took the level count from 65,536 to 4,294,967,296. Divide the big number by the small one to see how much more that is: $4{,}294{,}967{,}296 \div 65{,}536 = 65{,}536$ . So twice the storage bought 65,536 times as many levels. This is the reason bits are such a powerful currency, and it is also the reason that taking away a single bit hurts more than people expect.

The format(number_of_levels, ",") part inserts the thousands commas. Without it the last line would read 4294967296, which is harder to take in at a glance.

Now the reverse question, which is the one about the vocabulary. The next cell uses math.log2, the logarithm to base 2, which answers “two to the power of what gives me this number”.

# Cell 3. How many bits does a token id need?

vocabulary_size = 151936                       # real: the token count of Qwen2.5-0.5B-Instruct
bits_needed_exactly = math.log2(vocabulary_size)

print("vocabulary size:", format(vocabulary_size, ","))
print("bits needed, exactly:", round(bits_needed_exactly, 3))
print("17 bits holds:", format(2 ** 17, ","))
print("18 bits holds:", format(2 ** 18, ","))

vocabulary size: 151,936
bits needed, exactly: 17.213
17 bits holds: 131,072
18 bits holds: 262,144

Read the output as a short argument. The answer comes out at 17.213 bits, rounded to three decimal places, which is not a number of bits anyone can buy. Seventeen bits give 131,072 names, and the model needs 151,936, so 17 bits falls short by 20,864 names. Eighteen bits give 262,144 names, which covers it. So the answer is 18 bits, and 110,208 of those names go unused.

That wastage is normal and it is the price of a menu whose length can only double. It is also a small preview of the central problem of Chapter 7: a format has to cover the worst case, and everything that is not the worst case pays for the coverage.

6.2 From bits to bytes, and why a “1 GB model” is 1 GB¶

Intuition¶

Bits are a fine unit for thinking, and an inconvenient one for counting files. So computers group them in eights. Eight bits taken together are a byte. One byte holds $2^{8} = 256$ different values, which is roughly why a single byte was historically enough for one letter of text.

File sizes are counted in bytes, never in bits. When a download page says a model is 0.99 GB, it is counting bytes. This is the first place people trip, because network speeds are usually quoted in bits per second while file sizes are quoted in bytes, and the two differ by a factor of eight. This chapter only ever counts bytes.

Now put the two ideas together. A model is a pile of parameters. Every parameter is one number. Every number has to be written down using some fixed number of bits, and every parameter in a given file uses the same number of bits as every other one. So the size of the file is not a mystery and it is not negotiated. It is a multiplication with two inputs: how many parameters there are, and how many bytes each one costs.

Three storage choices cover almost everything you will meet.

FP32 spends 32 bits, that is 4 bytes, on each number. It is called full precision, and it is what a model is usually trained in.

FP16 spends 16 bits, that is 2 bytes. It is called half precision, and it is what every model in this course is shipped in. Half the bits, half the file.

INT8 spends 8 bits, that is 1 byte, and stores whole numbers rather than fractions. Chapter 7 explains how a fraction gets stored as a whole number. For now, all you need is the byte count.

The pattern is already visible. Each of those choices is half the one before it, so each one halves the file. There is no compression involved, no cleverness, and nothing hidden. The file halves because the multiplication halves.

The mathematics¶

Three short formulas do all of the work. Take them in order; each one feeds the next.

Formula 6.2, bytes per number

1. In words. To find out how many bytes one number costs, take the number of bits it uses and divide by eight, because eight bits make one byte.

2. The formula.

\text{bytes per number} = \frac{b}{8}

3. The symbols.

Symbol	How to say it out loud	What it means
$\text{bytes per number}$	“bytes per number”	the answer. It is allowed to be less than 1, and in Chapter 7 it will be.
$=$	“equals”	the two sides are the same number
$b$	“bee”	the bit width: how many bits one number uses
the fraction bar	“divided by”	divide the thing above the bar by the thing below it. See Toolkit 4.
$\div$	“divided by”	the same instruction as the fraction bar, written on one line. $\frac{16}{8}$ and $16 \div 8$ are two ways of writing the same thing, and this chapter uses whichever fits the line better.
8	“eight”	there are eight bits in one byte, so dividing by eight turns bits into bytes

4. Out loud. “The bytes per number is the bit width divided by eight.”

5. Worked, on the four bit widths this course uses.

At 32 bits: $32 \div 8 = 4$ bytes per number.

At 16 bits: $16 \div 8 = 2$ bytes per number.

At 8 bits: $8 \div 8 = 1$ byte per number.

At 4 bits: $4 \div 8 = 0.5$ bytes per number. Half a byte is a perfectly ordinary answer here. It means two numbers share one byte, four bits each.

6. Check it. Multiply your answer by 8 and you must land back on the bit width you started with. For the 4-bit row, $0.5 \times 8 = 4$ , which is correct. If you ever get a bytes-per-number figure larger than the bit width, you multiplied by 8 instead of dividing.

Formula 6.3, the size of a model in bytes

1. In words. Multiply how many parameters the model has by how many bytes each parameter costs. That is the size of the file.

2. The formula.

S = N \times \frac{b}{8}

3. The symbols.

Symbol	How to say it out loud	What it means
$S$	“capital ess”	the answer: the size of the stored weights, in bytes
$=$	“equals”	the two sides are the same number
$N$	“capital en”	the parameter count. $N = 494{,}032{,}768$ for `Qwen2.5-0.5B-Instruct`.
$\times$	“times”	multiply. See Toolkit 3 for the four ways this gets written.
$b$	“bee”	the bit width: how many bits one parameter uses
8	“eight”	eight bits in a byte, the same 8 as in Formula 6.2
the fraction bar	“divided by”	divide the thing above the bar by the thing below it
$\dfrac{b}{8}$	“b over eight”	the bytes per number, straight from Formula 6.2

4. Out loud. “The size in bytes is the number of parameters, multiplied by the bit width divided by eight.”

5. Worked, at 16 bits, on a real model. Qwen2.5-0.5B-Instruct has $N = 494{,}032{,}768$ parameters. That count is real, from lab/out/we3_params_quant.json.

Step 1, find the bytes per number using Formula 6.2.

\frac{16}{8} = 2 \text{ bytes per parameter}

Step 2, multiply the parameter count by that.

494{,}032{,}768 \times 2 = 988{,}065{,}536 \text{ bytes}

6. Check it. At 2 bytes each, the byte count has to be exactly twice the parameter count. It is: 494 million doubled is 988 million. And the answer must get smaller when $b$ gets smaller. At 8 bits you would get $494{,}032{,}768 \times 1 = 494{,}032{,}768$ bytes, which is half of the 16-bit answer, as it should be.

Formula 6.4, bytes into gigabytes

1. In words. A gigabyte is a thousand million bytes, so divide your byte count by a thousand million.

2. The formula.

G = \frac{S}{1{,}000{,}000{,}000}

3. The symbols.

Symbol	How to say it out loud	What it means
$G$	“capital gee”	the answer: the size in gigabytes
$=$	“equals”	the two sides are the same number
$S$	“capital ess”	the size in bytes, from Formula 6.3
the fraction bar	“divided by”	divide the thing above by the thing below
$1{,}000{,}000{,}000$	“one thousand million”, or “one billion”	how many bytes are in one gigabyte under the convention this book uses. It can also be written 10⁹, which is covered in Toolkit 17.

4. Out loud. “The size in gigabytes is the size in bytes divided by one thousand million.”

5. Worked, on the same model.

988{,}065{,}536 \div 1{,}000{,}000{,}000 = 0.988065536

Rounded to three decimal places, following Toolkit 16, that is 0.988 GB. The download bar rounded it further, to 0.99 GB.

That number is confirmed independently. lab/out/lab4_size_ladder.json records size_fp16_gb for this model as 0.988065536, which matches the hand arithmetic digit for digit.

6. Check it. Dividing by a thousand million moves the decimal point nine places to the left. Count them on 988,065,536 and you land on 0.988065536. If your answer came out as 988.065536, you divided by a million instead of a thousand million, which is three places short.

Putting all three formulas together gives the table this chapter was built to produce. Every figure in it is the multiplication above, done four times.

Format	Bit width $b$	Bytes per parameter	Size in bytes	Size
FP32	32	4.0	1,976,131,072	1.976 GB
FP16	16	2.0	988,065,536	0.988 GB
INT8	8	1.0	494,032,768	0.494 GB
4-bit, ideal	4	0.5	247,016,384	0.247 GB

Those four sizes are real, and they appear in _research/00-lab-verified-findings.md in the section on parameters. The FP16 row also matches size_fp16_gb in lab/out/lab4_size_ladder.json exactly.

The bottom row carries a warning, and the chapter will not hide it. The 0.247 GB figure is what 4 bits would cost if four bits were all you had to store. In a real 4-bit model there is a little extra bookkeeping to store alongside the weights, so the file is somewhat larger. Worked example 6.5 at the end of this chapter works out the real figure, and Chapter 7 explains what the bookkeeping is for. The word ideal in that row is doing honest work.

Why it is called floating point

This is background, not something you need in order to do the arithmetic in this chapter.

A floating point format splits its bits into three jobs: one bit records whether the number is positive or negative, a group of bits records roughly how big the number is, and the rest record the leading digits. The name comes from the middle group: the decimal point is allowed to float to wherever the size bits say it should go, instead of sitting in one fixed place. That is what lets a single format hold both 0.0000169, which is the size of the average of the real weight matrix in this chapter, and 1.2265625, which is its largest magnitude. Both are real, from W_stats in lab/out/we3_params_quant.json.

These are definitions of the format, set by a published standard, and they are not measurements. This chapter does not take any format apart bit by bit, because you do not need that in order to answer the question the chapter actually asks: how far apart are the values, and how much does a number move when it is pushed to the nearest one?

Worked example 6.1, how big is the 1.5B model at FP16?

Qwen2.5-1.5B-Instruct has $N = 1{,}543{,}714{,}304$ parameters. That count is real, from lab/out/lab4_size_ladder.json. How big is the file at FP16?

Step 1, bytes per parameter at 16 bits, using Formula 6.2.

16 \div 8 = 2

Step 2, multiply, using Formula 6.3.

1{,}543{,}714{,}304 \times 2 = 3{,}087{,}428{,}608 \text{ bytes}

Step 3, convert to gigabytes, using Formula 6.4.

3{,}087{,}428{,}608 \div 1{,}000{,}000{,}000 = 3.087428608

Answer: 3.087 GB. The lab recorded size_fp16_gb for this model as 3.087428608, so the hand arithmetic reproduces the measurement exactly.

Check it. This model has about three times the parameters of the 0.5B model, and its file is about three times as large: 3.087 against 0.988, and $3.087 \div 0.988 = 3.12$ , rounded to two decimal places. Now do the same division on the parameter counts: $1{,}543{,}714{,}304 \div 494{,}032{,}768 = 3.12$ , to two decimal places again. The ratio of the file sizes is the ratio of the parameter counts, because the bytes per parameter is the same for both.

Worked example 6.2, will the 3B model fit a 4 GB graphics card?

Qwen2.5-3B-Instruct has $N = 3{,}085{,}938{,}688$ parameters, which is real, from lab/out/lab4_size_ladder.json. A student laptop with a 4 GB graphics card wants to load it at FP16. Does it fit?

Step 1, bytes per parameter at 16 bits: $16 \div 8 = 2$ .

Step 2, multiply: $3{,}085{,}938{,}688 \times 2 = 6{,}171{,}877{,}376$ bytes.

Step 3, convert: $6{,}171{,}877{,}376 \div 1{,}000{,}000{,}000 = 6.171877376$ .

Answer: 6.172 GB, so it does not fit. The weights alone are more than half as large again as the whole card, before the model has stored a single thing about the text you gave it.

Now ask the same question of the 1.5B model, which is 3.087 GB from Worked example 6.1. That one does fit inside 4 GB, with about 0.9 GB spare.

Why this matters beyond the arithmetic. That single comparison is one half of this course’s central argument, and Chapter 7 develops it. The same multiplication that tells you a file size also tells you which students can run which models. A 4 GB card is an ordinary piece of student hardware. The line between 3.087 and 6.172 is not a matter of taste. It is a wall, and it is made of arithmetic you can now do yourself.

Try it 6.2

A model has exactly 2,000,000,000 parameters. This count is made up for practice, chosen because it is easy to work with.

How large is it at FP16, in gigabytes?
How large is it at INT8?
How large would it be at an ideal 4 bits?
The download page says the file is 4.0 GB. Which format is it stored in?

Python¶

The cell below does the whole table in one pass. It starts by reading the parameter count out of the lab’s own results file rather than typing it in, which means the number in your output came from the measurement and not from somebody’s memory.

# Cell 4. The size of a real model at four different bit widths.

# json.load opens the file and turns its contents into a Python dictionary, which is a
# collection of labelled values you look things up in by name.
measured_parameters = json.load(open(path_to_the_measured_file))

# "total_params" is the label the lab script wrote the parameter count under.
parameter_count = measured_parameters["total_params"]
print("parameter count:", format(parameter_count, ","))
print()

# The four storage choices this course uses, widest first.
bit_widths_for_storage = [32, 16, 8, 4]

for bits_per_parameter in bit_widths_for_storage:
    bytes_per_parameter = bits_per_parameter / 8          # Formula 6.2
    size_in_bytes = parameter_count * bytes_per_parameter  # Formula 6.3
    size_in_gigabytes = size_in_bytes / 1000000000         # Formula 6.4
    print(bits_per_parameter, "bits per parameter =", bytes_per_parameter, "bytes;",
          format(int(size_in_bytes), ","), "bytes;", round(size_in_gigabytes, 3), "GB")

parameter count: 494,032,768

32 bits per parameter = 4.0 bytes; 1,976,131,072 bytes; 1.976 GB
16 bits per parameter = 2.0 bytes; 988,065,536 bytes; 0.988 GB
8 bits per parameter = 1.0 bytes; 494,032,768 bytes; 0.494 GB
4 bits per parameter = 0.5 bytes; 247,016,384 bytes; 0.247 GB

Take the output one line at a time.

The first line prints 494,032,768. That value was not typed into this cell. It was read out of lab/out/we3_params_quant.json, which was written by a script that loaded the real model and counted. If somebody swapped the model, this number would change on its own, and so would every line under it. That is the shape every piece of code in this book tries to have.

The four lines that follow are the table from the previous section, and they match it exactly. 1.976, then 0.988, then 0.494, then 0.247. Read down the gigabyte column and check the halving: each figure is half the one above it, because each bit width is half the one above it.

Two small pieces of Python are worth naming. The single slash / is division, and it always gives a decimal answer, which is why 4 / 8 prints as 0.5 and not as 0. And round(x, 3) cuts a number to three decimal places for display, using exactly the rule from Toolkit 16. It changes what is printed; it does not change the value stored in the variable.

One habit to take from this cell: the code never writes 0.988 anywhere. It writes the multiplication that produces 0.988. If a printed number is a typed constant, it can be wrong without anything complaining. If it is a computed result, it has to be wrong on purpose.

Try one change and watch what happens. Replace the parameter count with 1,543,714,304, which is the real count for Qwen2.5-1.5B-Instruct, and run the cell again. Every one of the four lines moves, and the FP16 line becomes 3.087 GB, which is Worked example 6.1. Nothing else in the cell has to change, because nothing else in the cell knows which model it is describing. A cell built this way answers a question rather than reciting an answer, and you can point it at any model whose parameter count you know.

6.3 The ruler: the levels a format can write down¶

Intuition¶

Go back to the fruit stand. The scale reads to two decimal places, so the readings it can show are 0.00, 0.01, 0.02, and so on up to whatever its maximum is. Those readings are evenly spaced, one hundredth of a pound apart. There is nothing between 3.14 and 3.15. As far as that scale is concerned, no such weight exists.

Now picture the readings as marks on a ruler laid out along a line. Evenly spaced ticks. Every tick is a value the scale can show. The space between two neighbouring ticks is a place where the scale has nothing to say.

That picture is exactly right for a computer storing model weights, with two changes.

The first change is that the ruler runs both ways from zero, because weights can be negative. In the real weight matrix this chapter uses, the values run from -1.2265625 up to +1.171875. Both figures are real, from W_stats in lab/out/we3_params_quant.json. So the ruler has zero in the middle, ticks going right for positive values, and ticks going left for negative ones.

The ruler is the same length on both sides, so it has to be long enough for whichever end reaches further from zero. Throw away the minus signs and compare the two sizes: 1.2265625 against 1.171875. The first is larger, so the ruler has to reach 1.2265625 in both directions. That is where the $M = 1.2265625$ used in the rest of this chapter comes from, and it is why the most negative weight, not the most positive one, is the one that sets it.

The second change is that the number of ticks is not free. It is $2^{b}$ , from Formula 6.1, and that is the whole budget. If the ruler has to stretch from -1.2265625 to +1.2265625, and you only have a budget of 16 levels because you are at 4 bits, then those 16 levels have to cover a span of $1.2265625 + 1.2265625 = 2.453125$ . They will be far apart.

This is the sentence the whole chapter turns on. The number of ticks is set by the bits. How far apart they sit is set by the bits and by how far the ruler has to stretch. Two different formats can both use 4 bits and have wildly different precision, because one of them is covering a much narrower span with the same 16 levels.

There is one more bookkeeping detail, and it is worth getting right once rather than being confused by it later. Of the $b$ bits, one is spent recording whether the number is positive or negative. That leaves $b - 1$ bits for the size of the number. And one of the values those bits can express has to be zero itself, because a format that cannot write down zero is useless. So the number of whole steps you get on each side of zero is $2^{b-1} - 1$ , not $2^{b}$ .

At 8 bits that is $2^{7} - 1 = 127$ steps on each side. At 4 bits it is $2^{3} - 1 = 7$ steps on each side. Seven steps to the right of zero, seven to the left, plus zero in the middle. Fifteen usable ticks out of a 16-level budget, with one label going spare.

The mathematics¶

Formula 6.5, the gap between neighbouring levels

1. In words. Take the largest size the ruler has to reach, and share it out evenly among the steps you can afford on one side of zero. What each step is worth is the gap.

2. The formula.

g = \frac{M}{2^{\,b-1} - 1}

3. The symbols.

Symbol	How to say it out loud	What it means
$g$	“gee”	the answer: the gap, the distance from one tick to the next
$=$	“equals”	the two sides are the same number
$M$	“capital em”	the largest magnitude the ruler has to reach, that is, the biggest number in the group with its minus sign thrown away. See Toolkit 15.
$b$	“bee”	the bit width, how many bits you spend per number
$b - 1$	“bee minus one”	one bit goes to the plus-or-minus sign, so $b - 1$ bits are left to say how big the number is
$2^{\,b-1}$	“two to the bee minus one”	2 multiplied by itself $b-1$ times: how many sizes those bits can express
$-\,1$	“minus one”	subtract one, because one of those values has to be zero itself
the fraction bar	“divided by”	divide the thing above the bar by the thing below it

4. Out loud. “The gap is the largest magnitude, divided by the number of whole steps that fit on each side of zero.”

5. Worked, on a real weight matrix, at 8 bits. The matrix is model.layers[0].self_attn.q_proj.weight from Qwen2.5-0.5B-Instruct. It is 896 rows by 896 columns, which is $896 \times 896 = 802{,}816$ numbers, and its most negative value is -1.2265625. So the largest magnitude is $M = 1.2265625$ . Those figures are real, from lab/out/we3_params_quant.json under W_stats.

Step 1, work out $b - 1$ . With $b = 8$ , that is $8 - 1 = 7$ .

Step 2, raise 2 to that power. $2^{7} = 2 \times 2 \times 2 \times 2 \times 2 \times 2 \times 2$ . Doubling up from 2: 2, 4, 8, 16, 32, 64, 128. So $2^{7} = 128$ .

Step 3, subtract 1. $128 - 1 = 127$ . There are 127 whole steps on each side of zero.

Step 4, divide.

g = 1.2265625 \div 127 = 0.00965797

That division does not stop. Carried out further it reads 0.009657972440944882, which is the value the lab’s own script recorded as the 8-bit scale. Eight decimal places is all this chapter displays, because eight places fit on a phone screen and are plenty for every comparison made here. Where the ninth place changes an answer, the text says so at that spot.

Worked again, at 4 bits.

Step 1, $4 - 1 = 3$ .

Step 2, $2^{3} = 2 \times 2 \times 2 = 8$ .

Step 3, $8 - 1 = 7$ .

Step 4, $g = 1.2265625 \div 7 = 0.17522321$ , again rounded to eight decimal places. Carried further it reads $0.175223214285714\ldots$ , and the digits never stop.

6. Check it. Multiply the gap by the number of steps and you must land back on $M$ . At 8 bits, $0.00965797 \times 127 = 1.22656219$ . Set that beside $M = 1.2265625$ and the two agree to six decimal places. The tail differs because the gap was cut to eight places before it was multiplied by 127, so the check lands as close as the rounding allows and no closer. It checks out. The gap must also always be positive, and it must get larger as $b$ gets smaller, because fewer ticks spread over the same ruler have to sit further apart. Compare the two answers: 0.00965797 at 8 bits, and 0.17522321 at 4 bits. Divide the second by the first: $0.17522321 \div 0.00965797 = 18.14$ , rounded to two decimal places. So the gap grew by a factor of about 18 when four bits were taken away. A factor is what you multiply by, so “a factor of 18” means “18 times as large”.

Here is Formula 6.5 worked at every bit width this course uses, on the same real matrix, with $M = 1.2265625$ throughout. The gaps in the last column are rounded to seven decimal places, which is why the 8-bit entry reads 0.0096580 here and 0.00965797 in the worked example above. Both are the same number, shown to a different number of places.

Bit width $b$	$2^{\,b-1}$	Steps each side, $2^{\,b-1} - 1$	Gap $g$
8	128	127	0.0096580
6	32	31	0.0395665
4	8	7	0.1752232
3	4	3	0.4088542
2	2	1	1.2265625

Read the bottom row and let it land. At 2 bits there is one step on each side of zero. The entire list of values that format can write down is -1.2265625, 0, and +1.2265625. A matrix of 802,816 numbers, most of them small, has to be squeezed onto those three values. That is not a subtle loss of quality. That is the format having almost nothing to say.

Worked example 6.3, a ruler you can draw

The numbers here are made up for practice, so that the ruler is small enough to draw on paper.

You have 3 bits and you need to store numbers whose largest magnitude is $M = 10$ .

Step 1, $b - 1 = 3 - 1 = 2$ .

Step 2, $2^{2} = 2 \times 2 = 4$ .

Step 3, $4 - 1 = 3$ steps on each side of zero.

Step 4, $g = 10 \div 3 = 3.3333$ . That division never finishes; the 3s repeat forever, so 3.3333 is the answer rounded to four decimal places. The ticks below are rounded the same way, which is why three steps of 3.3333 read as 9.9999 rather than landing dead on 10.

So the entire list of values this format can write down is:

-10, \quad -6.6667, \quad -3.3333, \quad 0, \quad 3.3333, \quad 6.6667, \quad 10

Seven ticks. Draw them on a line. Now try to store the number 5 on that ruler. The nearest tick is 6.6667, and the one below is 3.3333. Five is 1.6667 away from the first and 1.6667 away from the second, so it sits exactly halfway between two ticks and is as badly served as a number can be by this format.

Check it. Count the ticks: three on the left, zero, three on the right, which is seven. Three bits give a budget of $2^{3} = 8$ levels, and seven of them got used. One label is spare, exactly as Section 6.3 said it would be.

Try it 6.3

Use the same real matrix, with $M = 1.2265625$ .

Work out the gap at 6 bits, showing every step.
The lab recorded the 6-bit scale for this matrix as 0.039566532258064516 in lab/out/we3_params_quant.json. Does your answer match?
Without computing anything, say whether the gap at 5 bits is bigger or smaller than the gap at 6 bits, and why.

Python¶

Two cells. The first computes the gap from scratch. The second checks it against what the lab actually recorded, which is the kind of check this book does at every opportunity.

# Cell 5. The gap between levels, at five bit widths, on one real weight matrix.

# The largest magnitude in that matrix. Real: W_stats.min in lab/out/we3_params_quant.json
# is -1.2265625, and the magnitude is that value with the minus sign thrown away.
largest_magnitude_in_matrix = 1.2265625

bit_widths_for_ruler = [8, 6, 4, 3, 2]

for bit_width in bit_widths_for_ruler:
    levels_for_size = 2 ** (bit_width - 1)          # one bit goes to the sign, so b - 1 are left
    steps_each_side = levels_for_size - 1           # one value has to be zero itself
    gap_between_levels = largest_magnitude_in_matrix / steps_each_side   # Formula 6.5
    print(bit_width, "bits: steps each side of zero =", steps_each_side,
          " gap =", round(gap_between_levels, 7))

8 bits: steps each side of zero = 127  gap = 0.009658
6 bits: steps each side of zero = 31  gap = 0.0395665
4 bits: steps each side of zero = 7  gap = 0.1752232
3 bits: steps each side of zero = 3  gap = 0.4088542
2 bits: steps each side of zero = 1  gap = 1.2265625

The five gap values are the ones in the table above. The step counts are 127, 31, 7, 3 and 1, which halve and lose one each time you take a bit away, because $2^{b-1} - 1$ is doing exactly that.

Look at the 2-bit line once more. One step on each side. Everything in a matrix of 802,816 numbers is going to be shoved onto one of three values.

Now the check. The lab’s own script computed a scale for each bit width when it quantized this matrix for real, and saved it. If our arithmetic is right, our gap and its scale are the same number.

# Cell 6. Does our hand arithmetic match what the lab actually computed?

for bit_width in bit_widths_for_ruler:
    steps_each_side = 2 ** (bit_width - 1) - 1
    gap_between_levels = largest_magnitude_in_matrix / steps_each_side
    # The lab stored its scales under string keys, so str(bit_width) turns 8 into "8".
    measured_scale = measured_parameters["quant"][str(bit_width)]["scale"]
    print(bit_width, "bits: our gap =", round(gap_between_levels, 7),
          " lab scale =", round(measured_scale, 7),
          " difference =", round(abs(gap_between_levels - measured_scale), 12))

8 bits: our gap = 0.009658  lab scale = 0.009658  difference = 0.0
6 bits: our gap = 0.0395665  lab scale = 0.0395665  difference = 0.0
4 bits: our gap = 0.1752232  lab scale = 0.1752232  difference = 0.0
3 bits: our gap = 0.4088542  lab scale = 0.4088542  difference = 0.0
2 bits: our gap = 1.2265625  lab scale = 1.2265625  difference = 0.0

Every difference is 0.0 to twelve decimal places. The formula you worked by hand and the quantizer that ran on the real weights are doing the same arithmetic.

That is worth more than it looks. A formula in a textbook is a claim. A formula that reproduces, to twelve decimal places, what a separate program computed from real model weights is a checked claim. When Chapter 7 reports that per-tensor quantization at 4 bits gives an average error worth 46.6% of the weights’ spread, you will be able to trace that number back through this same gap, computed by this same line of arithmetic.

Two words in that sentence are worth unpacking now. Per-tensor means one ruler for a whole weight matrix, which is the scheme this chapter has been building; a tensor is what a weight matrix is called once it may have more than two directions to it, and for this chapter you may read “tensor” as “matrix”. Spread means the standard deviation of the matrix, which is one number saying how far a typical weight sits from the average of all of them, and which Toolkit 10, Formula 9 builds from nothing. You are never asked to compute a standard deviation in this chapter. For this matrix the spread is 0.066741, which is real, from sd under W_stats in lab/out/we3_params_quant.json.

The 46.6% is then one division and one multiplication, and you can do both now. The average error at 4 bits with one ruler for the whole matrix is 0.03109840, which is real, from lab/out/we3b_quant.json. Divide it by the spread: $0.03109840 \div 0.066741 = 0.46596$ , to five decimal places. Multiply by 100 to turn that decimal into a percentage: $0.46596 \times 100 = 46.596$ , which is $46.6\%$ to one decimal place. Reporting an error as a share of the spread is a way of asking “is this error large compared with how much the weights vary in the first place?”, and Chapter 7 does the asking.

The abs(...) in that cell takes the absolute value, which is the size of a number with its minus sign removed. It is there because a difference could come out negative depending on which way round you subtract, and what we care about is how far apart the two values are, not which is larger. Toolkit 15 has it in full.

One more detail in that cell is worth naming, because it catches people. The lab wrote its results under the keys "8", "6", "4" and so on, in quotation marks, which makes them text rather than numbers. The loop variable bit_width holds the number 8, not the text "8", and those are different things to Python. str(bit_width) converts the number into the text, which is why the lookup finds anything at all. Leaving str(...) out gives an error that names a key you can see in the file, which is a confusing message until you know this.

6.4 Rounding to the nearest level, and the error that makes¶

Intuition¶

The scale at the fruit stand reads 3.14 for grapes that truly weigh 3.14159 pounds. Something was lost. How much? The difference: $3.14159 - 3.14 = 0.00159$ pounds. That is the rounding error, and it is a real quantity you can compute, not a vague sense of imprecision.

Notice what sets its size. It is not the number of digits on the display in any direct way. It is the spacing between the readings the display can show. The readings are one hundredth of a pound apart, and the true weight fell somewhere inside one of those hundredth-of-a-pound gaps. Wherever inside the gap it fell, the nearest reading is at most half a gap away, because if it were further than half a gap from the nearest reading, some other reading would be nearer.

That last sentence is the most useful thing in this chapter, so here it is on its own.

Rounding to the nearest level can never be wrong by more than half a gap.

It is a guarantee, not an average. No number, anywhere, ever, rounds by more than half a gap. And it explains why “how many bits” is the wrong question to ask on its own. The right question is “how big is the gap”, because the gap is the error.

There is one more thing to see, and it is the reason Chapter 7 exists. A number whose size is smaller than half a gap rounds to zero. Not to something small. To exactly zero. It vanishes.

The real weight matrix in this chapter has a typical weight of about -0.027. Its gap at 4 bits is 0.1752232, and half of that is 0.0876116. Since 0.027 is smaller than 0.0876, that weight rounds to zero and is erased. The model does not get a slightly worse version of that weight. It gets no weight at all. Hold on to that, because it is exactly what the lab measured, and it is the failure Chapter 7 diagnoses and fixes.

The mathematics¶

Two formulas. The first says how to round to the nearest level. The second says how wrong that can be.

Formula 6.6, snapping a value to the nearest level

1. In words. Work out how many gaps your number sits away from zero, round that count to a whole number, and then walk out that many gaps from zero. Where you land is the stored value.

2. The formula.

k = \operatorname{round}\!\left(\frac{v}{g}\right), \qquad \hat{v} = k \times g

3. The symbols.

Symbol	How to say it out loud	What it means
$v$	“vee”	the original value you want to store, before anything is done to it
$=$	“equals”	the two sides are the same number
$g$	“gee”	the gap between levels, from Formula 6.5
the fraction bar	“divided by”	divide the thing above by the thing below
$\dfrac{v}{g}$	“v over g”	how many gaps your value sits away from zero. Usually not a whole number.
$\operatorname{round}$	“round”	round to the nearest whole number. 2.4 becomes 2, 2.6 becomes 3, -0.7 becomes -1. See Toolkit 16.
$k$	“kay”	the whole number that actually gets stored: which tick you landed on, counting out from zero
the comma	no sound	it separates two formulas that are done in order, first $k$ , then $\hat{v}$
$\hat{v}$	“v hat”	the stored value: what you get back when you read the number later
the hat	“hat”	it marks a number that stands in for another number, rather than being the real thing
$\times$	“times”	multiply
$\lvert \; \rvert$	“the size of”, or “the absolute value of”	a pair of upright bars around a number. They throw away its minus sign, so $\lvert -3 \rvert = 3$ . Used in Step 5 below to report the error as a size. See Toolkit 15.

4. Out loud. “The stored tick number is the value divided by the gap, rounded to a whole number; and the value you get back is that tick number multiplied by the gap.”

5. Worked, on a real weight, at 8 bits. The value is $v = -0.02697754$ . That is the median magnitude of the 802,816 numbers in the real weight matrix, carrying a minus sign. Median magnitude means this: throw away every minus sign, line all 802,816 sizes up from smallest to largest, and take the one in the middle. Half the matrix is closer to zero than this and half is further away. It is real, and it is recorded in _research/00-lab-verified-findings.md, section 5.

Step 1, write down the gap at 8 bits, from Formula 6.5: $g = 0.00965797$ .

Step 2, divide the value by the gap.

-0.02697754 \div 0.00965797 = -2.79329

Step 3, round to the nearest whole number. The value -2.79329 sits between -3 and -2, and it is nearer to -3. So $k = -3$ . The stored number is the whole number -3.

Step 4, multiply back out to get the stored value.

\hat{v} = -3 \times 0.00965797 = -0.02897391

Step 1 wrote the gap out to eight decimal places, to keep that multiplication short enough to do on a phone. Carry the gap one place further, to 0.009657972, and the same multiplication gives -0.02897392 instead. That is the figure the lab recorded, it is the figure Cell 7 prints below, and it is the one this chapter carries forward. Common mistake 6 at the end of the chapter is exactly this: round once, at the end, not in the middle.

Step 5, work out how far the value moved. The upright bars mean “throw away the minus sign”, so what comes out is a size and never a negative number.

\lvert -0.02697754 - (-0.02897392) \rvert = 0.00199638

Worked again, at 4 bits. Same value, different gap.

Step 1, the gap at 4 bits, from Formula 6.5: $g = 0.17522321$ .

Step 2, $-0.02697754 \div 0.17522321 = -0.15396$ .

Step 3, round. The value -0.15396 sits between -1 and 0, and it is far nearer to 0. So $k = \mathbf{0}$ .

Step 4, $\hat{v} = 0 \times 0.17522321 = \mathbf{0}$ .

Step 5, the error is the whole weight: 0.02697754. The number is gone.

6. Check it. The tick number $k$ must be a whole number, never a fraction. The stored value $\hat{v}$ must have the same sign as the original, or be exactly zero. And the error must never be bigger than half the gap, which Formula 6.7 comes to next. At 8 bits, half the gap is $0.00965797 \div 2 = 0.004828985$ , and the error 0.00199638 is comfortably under it. If your error comes out larger than half a gap, you rounded the wrong way in Step 3.

Formula 6.7, the largest error rounding can cause

1. In words. However badly placed your number is, the nearest level is never more than halfway across a gap, so the worst error you can ever suffer is half a gap.

2. The formula.

E_{\max} = \frac{g}{2}

3. The symbols.

Symbol	How to say it out loud	What it means
$E_{\max}$	“E sub max”	the answer: the largest rounding error that is possible with this gap
the small raised “max”	“max”, short for maximum	it is a label, not a calculation. It says which $E$ this is.
$=$	“equals”	the two sides are the same number
$g$	“gee”	the gap between levels, from Formula 6.5
the fraction bar	“divided by”	divide the thing above by the thing below
2	“two”	divide by two, because half of a gap is as far as the nearest tick can ever be

4. Out loud. “The largest possible error is the gap divided by two.”

5. Worked, at every bit width, on the real matrix. Take each gap from Section 6.3 and halve it.

Halving a number is the same as dividing it by two, so each line below is one division.

At 8 bits: $0.0096580 \div 2 = 0.0048290$ .

At 6 bits: $0.0395665 \div 2 = 0.01978325$ .

At 4 bits: $0.1752232 \div 2 = 0.0876116$ .

At 3 bits: $0.4088542 \div 2 = 0.2044271$ .

At 2 bits: $1.2265625 \div 2 = 0.61328125$ .

Each of those five divisions is exact, with no rounding done at the end. The gaps going in were the seven-decimal-place figures from Section 6.3, so each half carries whatever rounding its gap already carried and nothing new. That is why the 6-bit and 2-bit answers run to eight decimal places: halving a number that ends in a 5 adds one more place.

6. Check it, against the real run. This is the best check in the chapter, because it does not rely on the formula at all. The lab quantized all 802,816 weights in that matrix for real and recorded the single largest error it saw, under max_abs_err in lab/out/we3_params_quant.json. If Formula 6.7 is right, the measured worst error should land on half a gap, because in a matrix of 802,816 numbers something will land near the middle of a gap.

Both columns below carry nine decimal places, because that is where the two start to differ. The left column was worked from the gap at full precision, not from the seven-place gaps listed above, so that the rounding on this page cannot be blamed for any disagreement.

Bit width	Half a gap, from Formula 6.7	Largest error the lab measured
8	0.004828986	0.004828990
6	0.019783266	0.019783258
4	0.087611607	0.087611616
3	0.204427083	0.204427063
2	0.613281250	0.613281250

The first four rows agree to seven decimal places. Round both columns to seven places and the 8-bit row reads 0.0048290 twice, the 6-bit row reads 0.0197833 twice, the 4-bit row reads 0.0876116 twice, and the 3-bit row reads 0.2044271 twice. The last row agrees exactly, all the way down.

That last row is the clue to where the small disagreements come from, and the answer is not what you might guess.

At every one of the five bit widths, the weight that ends up worst off is the same weight, 0.61328125. That is exactly half of $M = 1.2265625$ , the largest magnitude in the matrix. A value sitting exactly halfway between two rungs is the worst case by construction, so this is not a coincidence.

Work the five errors in exact fractions, with no computer arithmetic anywhere, and each one comes out exactly half a gap. Not nearly. Exactly. So the way the weights are stored cannot be what breaks the tie, because in exact arithmetic there is no tie to break.

What differs is the arithmetic the script does on the way to the answer. To snap a weight the script computes $k \times g$ , the rung number times the gap. At 2 bits the rung number is $k = 0$ , so there is no multiplication to do and the answer comes back exact, which is why the last row agrees to the final digit. At 8, 6, 4 and 3 bits the rung number is not zero, and multiplying a whole number by a gap that never terminates in binary loses a few billionths. That is the whole of the eighth and ninth decimal place.

It is worth saying plainly what this means. The disagreement is not the model being imprecise. It is our measurement of the model being imprecise, in the ninth decimal place, because of one multiplication. A course about rounding error should be able to find the rounding error in its own tools, and that is what just happened.

Half a gap is not an estimate. It is a ceiling, and a real run walked right up to it.

Worked example 6.4, the bit width where the weight disappears

The weight and the range of the matrix are real, from lab/out/we3_params_quant.json and the lab findings. The gaps at 8, 6, 4, 3 and 2 bits are the lab’s own measured scales from that same file. The 5-bit gap is not in the lab file, because the lab never ran a 5-bit setting. It is worked out here with Formula 6.5 from the same real $M = 1.2265625$ : $2^{5-1} - 1 = 16 - 1 = 15$ steps each side, and $1.2265625 \div 15 = 0.08177083$ .

Take the typical weight $v = -0.02697754$ and follow it down through the bit widths. The question is: at what point does it stop existing?

At 8 bits, $g = 0.00965797$ and half a gap is 0.00482899. The weight’s size, 0.02697754, is larger than half a gap, so it survives. It stores as -0.02897392.

At 6 bits, $g = 0.03956653$ and half a gap is 0.01978327. The weight’s size, 0.02697754, is still larger, so it survives. It stores as -0.03956653, which is one whole tick out from zero.

At 5 bits, $g = 0.08177083$ and half a gap is 0.04088542. Now 0.02697754 is smaller than half a gap. The nearest tick to the weight is zero itself. The weight stores as 0.0 and it is gone.

At 4, 3 and 2 bits, the gaps are larger still, so the weight remains gone.

Answer: it disappears at 5 bits, and every setting below that.

Why this is the headline. Five bits gives $2^{5} = 32$ levels, which sounds like plenty for storing a number near zero. It is not, because the 32 levels are not placed near zero. They are spread across a ruler that has to reach 1.2265625, and half of the matrix sits within 0.02697754 of zero. A format that cannot see a typical weight is not slightly lossy. It has deleted half the matrix.

Chapter 7 fixes this, and the fix is not more bits.

Try it 6.4

The gap at 4 bits on this matrix is $g = 0.17522321$ , which is real, from lab/out/we3_params_quant.json. The value in question 3 is real too. The value 0.4 in question 1 is made up for practice, chosen because it is easy to divide on a phone.

Store the value $v = 0.4$ on this ruler. Find $k$ , find $\hat{v}$ , and find the error.
Is that error under half a gap?
Store the value $v = 1.171875$ , which is the largest positive weight in the matrix. What is the error, and why is it that small?

Python¶

Two cells again. The first does the rounding. The second checks the half-a-gap guarantee against the real run.

# Cell 7. Round one real weight to the nearest level, at five bit widths.

# Real: the median magnitude of the 802,816 weights in that matrix, carrying its sign.
# Recorded in _research/00-lab-verified-findings.md, section 5.
typical_weight = -0.02697754

for bit_width in bit_widths_for_ruler:
    steps_each_side = 2 ** (bit_width - 1) - 1
    gap_between_levels = largest_magnitude_in_matrix / steps_each_side    # Formula 6.5
    level_number = round(typical_weight / gap_between_levels)             # Formula 6.6, first part
    snapped_weight = level_number * gap_between_levels                    # Formula 6.6, second part
    rounding_error = abs(typical_weight - snapped_weight)                 # how far it moved
    print(bit_width, "bits: level number =", level_number,
          " snapped =", round(snapped_weight, 8),
          " error =", round(rounding_error, 8))

8 bits: level number = -3  snapped = -0.02897392  error = 0.00199638
6 bits: level number = -1  snapped = -0.03956653  error = 0.01258899
4 bits: level number = 0  snapped = 0.0  error = 0.02697754
3 bits: level number = 0  snapped = 0.0  error = 0.02697754
2 bits: level number = 0  snapped = 0.0  error = 0.02697754

Read the level number column downwards, because that column is the story.

At 8 bits the weight sits 3 ticks out from zero, and it stores as -0.02897392, which is the same number the hand arithmetic produced in Formula 6.6. At 6 bits it sits 1 tick out. At 4 bits and below the level number is 0, which means the nearest tick is zero itself, and the stored value is exactly 0.0.

The error column says the same thing in a second way. It grows from 0.00199638 to 0.01258899, and then it stops growing and sits at 0.02697754 for the last three rows. It stops growing because once a number has been rounded to zero, it cannot be rounded any further away. The error is the whole weight, and that is the largest the error for this value can get.

This is the row that Chapter 7 is about. A 4-bit model built this way has thrown away its typical weights, kept only its large ones, and the arithmetic above shows exactly how it happened.

Now the guarantee. The next cell puts half a gap side by side with the largest error the lab actually measured across all 802,816 weights.

# Cell 8. Is the worst error really half a gap? Check against the measured run.

for bit_width in bit_widths_for_ruler:
    steps_each_side = 2 ** (bit_width - 1) - 1
    gap_between_levels = largest_magnitude_in_matrix / steps_each_side
    worst_possible_error = gap_between_levels / 2                        # Formula 6.7
    # What the lab actually saw, across all 802,816 weights in the matrix.
    measured_worst_error = measured_parameters["quant"][str(bit_width)]["max_abs_err"]
    print(bit_width, "bits: half a gap =", round(worst_possible_error, 9),
          " largest error the lab measured =", round(measured_worst_error, 9))

8 bits: half a gap = 0.004828986  largest error the lab measured = 0.00482899
6 bits: half a gap = 0.019783266  largest error the lab measured = 0.019783258
4 bits: half a gap = 0.087611607  largest error the lab measured = 0.087611616
3 bits: half a gap = 0.204427083  largest error the lab measured = 0.204427063
2 bits: half a gap = 0.61328125  largest error the lab measured = 0.61328125

Compare the two columns row by row. At 8 bits: 0.004828986 against 0.004828990. At 4 bits: 0.087611607 against 0.087611616. At 2 bits the two columns are identical.

Formula 6.7 was derived by thinking about ticks on a line. The right-hand column came from a program that rounded 802,816 real numbers and kept a record of the worst case. They agree to seven decimal places. That is the kind of agreement that tells you a formula is describing something real rather than being a tidy story.

The differences in the last decimal place are honest and they have a cause. The weights in the file are themselves stored in a format with a gap, so the lab’s arithmetic carries a small rounding error of its own before it starts. This book does not round those differences away to make the columns match. When two numbers that ought to be equal are not quite equal, the reason is worth printing.

The simulation¶

The simulation below is the picture this whole section has been describing. The dot is one real number, 0.02697754, the median magnitude of the 802,816 weights in that real matrix. The tick marks are the levels, the diamond is where the number gets rounded to, and the readouts give the gap, the stored value, and the error.

Two things to do with it, in this order.

First, start at 8 bits and drag Bits per number down one notch at a time, watching the Gap between levels readout. Each notch throws away half the levels, and the gap grows to a little more than twice what it was. By 5 bits the Snapped value reads 0.00000000, which is Worked example 6.4 happening in front of you.

Second, set the bits back to 4 and then drag Scale maximum all the way down to 0.0313. Nothing about the bits changed. Four bits, sixteen levels, exactly as before. The error falls to about 0.0002. That is Section 6.5.

6.5 Same bits, different rulers¶

Intuition¶

Two rulers sit on the table. Both have fifteen tick marks. Fifteen ticks leave fourteen spaces between them, one fewer than the number of ticks, in the same way that five fence posts in a row leave four gaps. One ruler is a metre long, which is 100 centimetres, so its ticks sit $100 \div 14 = 7.14$ centimetres apart, to two decimal places. The other ruler is a centimetre long, which is 10 millimetres, so its ticks sit $10 \div 14 = 0.71$ millimetres apart, again to two decimal places. Same number of ticks, and a hundredfold difference in what they can measure.

Nothing about the ticks changed. What changed is how far they had to stretch.

That is the second half of Formula 6.5 and it is the half people skip. The gap has two inputs, not one. The bottom of the fraction, $2^{b-1} - 1$ , is set by the bits. The top of the fraction, $M$ , is set by the largest number you have to cover. Halving the range does the same good as gaining a bit, and doubling the range does the same damage as losing one.

Now put a real number on it. The matrix in this chapter has a largest magnitude of 1.2265625 and a typical magnitude of 0.02697754. Divide one by the other: $1.2265625 \div 0.02697754 = 45.47$ , rounded to two decimal places. The biggest weight in that matrix is about 45 times the size of a typical one. And because the ruler has to reach the biggest one, every tick is spaced for the biggest one. The typical weights, which are the overwhelming majority, live crammed into the first one forty-fifth of the ruler, where there are barely any ticks at all.

This is not a small inefficiency. It is the reason a 4-bit model built the naive way is useless, and it is the reason a 4-bit model built a slightly different way is genuinely good. The difference between those two models is not the bit width. It is what each group of weights is allowed to use as its own $M$ .

That is the whole of Chapter 7 in one paragraph, and you now have the arithmetic to check it.

The mathematics¶

First, what one bit is worth. It is close to a doubling of the gap, and it is worth knowing that it is slightly more than a doubling, and why.

Formula 6.8, what dropping one bit does to the gap

1. In words. Compare the number of steps you have before and after, and the gap grows by whatever that comparison says. Losing a bit takes away a little more than half your steps, so the gap grows by a little more than double.

2. The formula.

\frac{g_{b-1}}{g_{b}} = \frac{2^{\,b-1} - 1}{2^{\,b-2} - 1}

3. The symbols.

Symbol	How to say it out loud	What it means
$g_{b}$	“g sub b”	the gap you get with $b$ bits
$g_{b-1}$	“g sub b minus one”	the gap you get with one bit fewer
the subscript	“sub”	a small label attached to a letter, saying which one of several. See Toolkit 2.
the fraction bar on the left	“divided by”	divide the new gap by the old one, giving the factor it grew by
$=$	“equals”	the two sides are the same number
$2^{\,b-1} - 1$	“two to the b minus one, minus one”	the steps each side of zero at the original bit width
$2^{\,b-2} - 1$	“two to the b minus two, minus one”	the steps each side of zero after losing a bit
the fraction bar on the right	“divided by”	divide the old step count by the new one

Both $M$ values cancel out, because the same $M$ appears on the top and the bottom when you divide the two gaps. That is why $M$ does not appear in this formula at all: the factor the gap grows by does not depend on how long the ruler is.

4. Out loud. “The factor the gap grows by is the number of steps you had, divided by the number of steps you are left with.”

5. Worked, going from 8 bits to 7 bits.

Step 1, steps at 8 bits: $2^{8-1} - 1 = 2^{7} - 1 = 128 - 1 = 127$ .

Step 2, steps at 7 bits: $2^{8-2} - 1 = 2^{6} - 1 = 64 - 1 = 63$ .

Step 3, divide.

127 \div 63 = 2.015873

So dropping from 8 bits to 7 bits makes the gap about 2.0159 times wider. Slightly more than double.

Worked again, going from 3 bits to 2 bits.

Step 1, steps at 3 bits: $2^{2} - 1 = 4 - 1 = 3$ .

Step 2, steps at 2 bits: $2^{1} - 1 = 2 - 1 = 1$ .

Step 3, divide: $3 \div 1 = 3$ .

Dropping from 3 bits to 2 bits makes the gap three times wider, not twice. The doubling rule breaks down badly at the bottom of the range, because the “minus one” is a large share of a small number.

6. Check it. Do it the long way and compare. At 8 bits the gap is $1.2265625 \div 127 = 0.00965797$ , and at 7 bits it is $1.2265625 \div 63 = 0.01946925$ . Both of those are rounded to eight decimal places. Divide the second by the first: $0.01946925 \div 0.00965797 = 2.0158739$ . Step 3 gave 2.015873, which written to the same seven places is 2.0158730.

Those two agree to five decimal places and then part company, and the reason is worth naming because it is the trap Common mistake 6 warns about. Step 3 divided two whole numbers, 127 by 63, and whole numbers carry no rounding. This check divided two gaps that had already been cut to eight decimal places, so the rounding in each one leaked into the answer. Carry the two gaps at full precision and the check lands on 2.0158730 exactly. When a check disagrees in the seventh decimal place, suspect rounding before you suspect the formula.

The factor must always be bigger than 2 and never smaller, because you are always dividing by a number slightly less than half of the one above it.

Now the second input, which is the range. Formula 6.5 has $M$ sitting on the top of the fraction, so the gap is directly proportional to $M$ . Two quantities are directly proportional when multiplying one of them by some number multiplies the other by that same number. Here it means: halve the range and you halve the gap, double the range and you double the gap, and the bits never move. Toolkit 12 builds that idea from nothing.

Here is that comparison. The weight being stored and the whole-matrix range of 1.2265625 are real. The narrow range of 0.03125 in the second row is made up for practice: it is a round number picked to sit close to where the typical weights actually live, and no block in a real quantizer will have exactly that $M$ . Both rows use 4 bits, so both rows have 7 steps on each side of zero.

What the ruler has to reach, $M$	Steps each side	Gap $g$	Where the typical weight lands	Error
1.2265625, the whole matrix	7	0.17522321	tick 0, value 0.0	0.02697754
0.03125, a narrow group	7	0.00446429	tick -6, value -0.02678571	0.00019183

The bit width is identical in those two rows. The error differs by a factor of about 141: $0.02697754 \div 0.00019183 = 140.6$ , rounded to one decimal place. And the ranges differ by a factor of $1.2265625 \div 0.03125 = 39.25$ , which is exact.

Read the second row once more. Four bits, sixteen levels, and the typical weight is stored to within two ten-thousandths. Four bits is not the problem. Asking four bits to cover a range 45 times wider than the numbers that actually need storing is the problem.

Worked example 6.5, what a real 4-bit file actually costs

The table in Section 6.2 said 4 bits gives 0.247 GB, and it flagged that figure as ideal. Here is the real one.

A working 4-bit quantizer does not use one ruler for the whole model. It cuts the weights into small blocks and gives each block its own $M$ , which is the fix Section 6.5 has been pointing at. This book’s blocks hold 32 weights each. But every block’s own $M$ has to be written down in the file too, and it is stored using 16 bits.

Step 1, work out each weight’s share of its block’s bookkeeping. One 16-bit number is shared by 32 weights.

16 \div 32 = 0.5 \text{ bits per weight}

Step 2, add that to the 4 bits spent on the weight itself.

4 + 0.5 = 4.5 \text{ bits per weight}

Step 3, turn bits into bytes, using Formula 6.2.

4.5 \div 8 = 0.5625 \text{ bytes per weight}

Step 4, multiply by the parameter count, using Formula 6.3.

494{,}032{,}768 \times 0.5625 = 277{,}893{,}432 \text{ bytes}

Step 5, convert to gigabytes, using Formula 6.4.

277{,}893{,}432 \div 1{,}000{,}000{,}000 = 0.277893432

Answer: 0.278 GB, against 0.247 GB for the ideal case.

Check it. The overhead is $0.5625 \div 0.5 = 1.125$ . Turning that decimal into a percentage takes two moves, both of them set out from scratch in Toolkit 11. First subtract the 1, because the 1 is the part you were paying already, leaving $1.125 - 1 = 0.125$ . Then multiply by 100, because “per cent” means “out of a hundred”, giving $0.125 \times 100 = 12.5$ , that is $12.5\%$ . So the block scales cost 12.5% on top of the weights.

What does that 12.5% buy? The lab measured it, so we do not have to guess. At 4 bits, the average error per weight is 0.03109840 with one ruler for the whole matrix and 0.00483112 with one ruler for every block of 32 weights. Both figures are real, from lab/out/we3b_quant.json, rounded to eight decimal places from the mean entries there. Divide the first by the second: $0.03109840 \div 0.00483112 = 6.44$ , rounded to two decimal places. The blocks cut the average error to under a sixth of what it was, for a 12.5% surcharge on the file. That is why every real 4-bit model is built this way. The 0.278 GB figure is also in section 4.6 of the formula appendix, worked the same way.

Try it 6.5

Dropping from 6 bits to 5 bits: by what factor does the gap grow? Show the steps.
A group of weights has $M = 0.25$ . At 4 bits, what is the gap, and what is the largest possible rounding error? These numbers are made up for practice.
Somebody says “our model uses 4-bit weights, so its error is fixed at 4-bit level.” What is wrong with that sentence?

Python¶

The first cell prints the gap at every bit width from 8 down to 2, so you can watch it grow.

# Cell 9. How the gap grows as bits are taken away, one at a time.

for bit_width in [8, 7, 6, 5, 4, 3, 2]:
    steps_each_side = 2 ** (bit_width - 1) - 1
    gap_between_levels = largest_magnitude_in_matrix / steps_each_side    # Formula 6.5
    print(bit_width, "bits: gap =", round(gap_between_levels, 7))

8 bits: gap = 0.009658
7 bits: gap = 0.0194692
6 bits: gap = 0.0395665
5 bits: gap = 0.0817708
4 bits: gap = 0.1752232
3 bits: gap = 0.4088542
2 bits: gap = 1.2265625

Read it downwards and you can see each gap is a bit more than twice the one above it. From 0.009658 to 0.0194692 is a bit more than double. From 0.4088542 to 1.2265625 is three times.

The 5-bit line is the one to note, because 0.0817708 halved is 0.0408854, and the typical weight of 0.02697754 is smaller than that. That is the line at which the weight disappears, which is Worked example 6.4.

The next cell computes the growth factor directly, with no division of gaps involved, using Formula 6.8.

# Cell 10. The factor the gap grows by, each time one bit is removed.

for bit_width in [8, 7, 6, 5, 4, 3]:
    steps_at_this_width = 2 ** (bit_width - 1) - 1      # steps each side before
    steps_one_bit_lower = 2 ** (bit_width - 2) - 1      # steps each side after
    gap_ratio = steps_at_this_width / steps_one_bit_lower
    print("dropping from", bit_width, "bits to", bit_width - 1,
          "multiplies the gap by", round(gap_ratio, 6))

dropping from 8 bits to 7 multiplies the gap by 2.015873
dropping from 7 bits to 6 multiplies the gap by 2.032258
dropping from 6 bits to 5 multiplies the gap by 2.066667
dropping from 5 bits to 4 multiplies the gap by 2.142857
dropping from 4 bits to 3 multiplies the gap by 2.333333
dropping from 3 bits to 2 multiplies the gap by 3.0

Every factor is above 2, and they get worse as you go down. Notice that the first line, 2.015873, matches the hand arithmetic in Formula 6.8 exactly, and that the last line is a clean 3.0.

That last line is why the bottom of the range behaves so badly. At 3 bits you have 3 steps each side. At 2 bits you have 1. Dropping a bit there does not halve your ticks; it removes two thirds of them.

Finally, the comparison that sets up Chapter 7: two different rulers, the same four bits.

# Cell 11. Four bits, two different ranges. The bits do not change. The error does.

wide_range_top = 1.2265625     # real: the largest magnitude in the whole matrix
narrow_range_top = 0.03125     # made up for practice: a narrow group, near where typical weights live
bit_width_for_both = 4
steps_each_side_for_both = 2 ** (bit_width_for_both - 1) - 1

wide_gap = wide_range_top / steps_each_side_for_both
narrow_gap = narrow_range_top / steps_each_side_for_both
print("4 bits, ruler top", wide_range_top, ": gap =", round(wide_gap, 8))
print("4 bits, ruler top", narrow_range_top, ": gap =", round(narrow_gap, 8))

wide_level = round(typical_weight / wide_gap)
narrow_level = round(typical_weight / narrow_gap)
print("typical weight snaps to level", wide_level, "on the wide ruler, value",
      round(wide_level * wide_gap, 8))
print("typical weight snaps to level", narrow_level, "on the narrow ruler, value",
      round(narrow_level * narrow_gap, 8))
print("error on the wide ruler  =", round(abs(typical_weight - wide_level * wide_gap), 8))
print("error on the narrow ruler =", round(abs(typical_weight - narrow_level * narrow_gap), 8))
print("ratio of the two ranges =", round(wide_range_top / narrow_range_top, 4))

4 bits, ruler top 1.2265625 : gap = 0.17522321
4 bits, ruler top 0.03125 : gap = 0.00446429
typical weight snaps to level 0 on the wide ruler, value 0.0
typical weight snaps to level -6 on the narrow ruler, value -0.02678571
error on the wide ruler  = 0.02697754
error on the narrow ruler = 0.00019183
ratio of the two ranges = 39.25

On the wide ruler the weight lands on level 0 and is erased. On the narrow ruler it lands on level -6, six ticks out from zero, and it stores as -0.02678571, which is within two ten-thousandths of the truth.

The bit_width_for_both variable is 4 on both sides of that comparison. Nothing about the storage budget changed between the two rows. What changed is the number the ruler had to reach, and it changed by a factor of 39.25.

That is the last idea this chapter has to hand over. Precision is not something you buy with bits alone. It is something you buy with bits and with keeping the range small, and the second one is usually cheaper. Chapter 7 shows you what a real quantizer does with that fact, measures what it costs in accuracy and in energy, and then asks who is on which side of the line it draws.

Common mistakes¶

These are the six things that go wrong most often with this material, and how to notice each one.

Thinking that doubling the bit width doubles the number of levels. It squares it. Going from 4 bits to 8 bits goes from 16 levels to 256 levels, which is sixteen times as many, not twice. What doubles the levels is adding one bit. Bits add; levels multiply. If your answer to “how many levels in 8 bits” is 32, you doubled the wrong thing.
Mixing up bits and bytes. A byte is eight bits, and file sizes are counted in bytes. FP16 is 16 bits, which is 2 bytes, not 16 bytes. The check: your bytes-per-parameter figure can never be larger than the bit width. If you get 16 bytes per parameter, you multiplied by 8 instead of dividing.
Believing that “4-bit” names an error. It does not. It names a storage budget. The error is the gap, and the gap is $M \div (2^{b-1} - 1)$ , which depends on the range as much as on the bits. Two 4-bit schemes in this chapter had errors 141 times apart. The only honest statement about the error of a scheme is a measured one.
Forgetting the minus one in $2^{b-1} - 1$ . One bit goes to the sign, and one value has to be zero itself. At 4 bits that leaves 7 steps each side, not 8 and not 16. The check: count your ticks. Seven left, zero, seven right is fifteen ticks from a 16-level budget, with one spare.
Reporting a number rounded to zero as “a small error”. When a weight rounds to zero, the error is the whole weight. The number did not get less accurate; it stopped existing. The check: if the level number $k$ comes out as 0 for a value that is not near zero, say so in words rather than printing an error figure and moving on.
Rounding in stages. Carry the full precision through a calculation and round once, at the end, for display. Rounding a gap to 0.0097 and then using 0.0097 in the next step builds error that was not there before. Toolkit 16 works a case where staged rounding lands on the wrong answer by a whole tenth.

What to remember¶

A bit is one yes-or-no answer, and $b$ bits name $2^{b}$ different values with nothing in between them. A model’s file size is one multiplication: the parameter count times the bytes spent on each parameter, which is why 494,032,768 parameters at 2 bytes each is 0.988 GB. The values a format can store sit on a ruler with an even gap between them, and that gap, $M \div (2^{b-1} - 1)$ , is what precision actually means. Rounding to the nearest level is never wrong by more than half a gap, and a number smaller than half a gap rounds to zero and is gone. The gap depends on the range as much as on the bits, and that single fact is the whole of Chapter 7.

Practice problems¶

Twenty-six problems in three tiers. Warm-up asks whether you can do the arithmetic. Practice asks whether you can apply it. Stretch asks whether you can reason with it.

Worked solutions are being added to the answers appendix as the problem set is finalised; the standing computation for this chapter, which is the file-size calculation, is worked in full there now.

Numbers marked real come from lab/out/we3_params_quant.json, lab/out/lab4_size_ladder.json, lab/out/we3b_quant.json or _research/00-lab-verified-findings.md. Numbers marked made up for practice were chosen to be easy to check on a phone.

Warm-up¶

How many different values fit in 3 bits? In 6 bits? In 12 bits? Show the doubling.
How many bytes is 24 bits? How many bits is 5 bytes?
A format spends 16 bits per number. How many bytes per number is that?
A format spends 4 bits per number. How many bytes per number is that? Explain why the answer is allowed to be less than 1.
Convert 2,500,000,000 bytes to gigabytes.
Convert 0.75 GB to bytes.
At 4 bits, how many whole steps fit on each side of zero? Show the three steps of the calculation.
At 6 bits, how many whole steps fit on each side of zero?
A ruler has to reach $M = 8$ and you have 3 bits. What is the gap? These numbers are made up for practice.
A gap is 0.05. What is the largest rounding error possible? These numbers are made up for practice.

Practice¶

Qwen2.5-0.5B-Instruct has 494,032,768 parameters (real). Work out its size in bytes and in gigabytes at FP32, at FP16 and at INT8. Show all three steps each time.
Qwen2.5-3B-Instruct has 3,085,938,688 parameters (real). Work out its size at FP16 and at INT8. Which of the two fits on a 4 GB graphics card?
A model file is 1.5 GB and the model has 1,500,000,000 parameters (made up for practice). How many bytes per parameter, and therefore how many bits, is it stored at?
Using the real matrix with $M = 1.2265625$ : compute the gap at 3 bits, showing every step, and then state the largest possible rounding error at that bit width.
Store the value $v = 0.9$ on the 4-bit ruler for that matrix, where $g = 0.17522321$ . Find $k$ , find $\hat{v}$ , and find the error. Confirm the error is under half a gap.
Store the value $v = -0.5$ on the same 4-bit ruler. Find $k$ , $\hat{v}$ and the error.
The typical weight in that matrix is -0.02697754 (real). Show, with arithmetic, that it survives at 6 bits and is erased at 5 bits.
Using Formula 6.8, find the factor the gap grows by when you drop from 5 bits to 4 bits. Confirm it by computing both gaps directly with $M = 1.2265625$ and dividing.
A group of weights has $M = 0.5$ and you have 5 bits (made up for practice). Compute the gap, then the largest possible error, then say whether a weight of size 0.01 survives.
A 4-bit scheme stores a 16-bit scale for every block of 64 weights, rather than every 32. Compute the bytes per weight, then the size of the 494,032,768-parameter model under that scheme. Compare with the 0.278 GB figure from Worked example 6.5.

Stretch¶

Explain in your own words, in no more than five sentences and with no formulas, why two different 4-bit schemes can have errors that differ by a factor of more than a hundred.
Here is the whole picture, not just one row. The left column is half a gap, computed with Formula 6.7 from the real $M = 1.2265625$ . The right column is the measured largest rounding error, real, from max_abs_err in lab/out/we3_params_quant.json.
bits half a gap measured worst error
8 0.004828986 0.004828990
6 0.019783266 0.019783258
4 0.087611607 0.087611616
3 0.204427083 0.204427063
2 0.613281250 0.613281250
(a) For each row, say whether the measured value sits above, below, or exactly on half a gap. (b) A classmate says the measured worst case should always land a little under the ceiling, because a real weight is unlikely to sit exactly halfway between two rungs. Use your answer to (a) to show that this explanation cannot be right. (c) The disagreements are all in the ninth decimal place and they change sign. What does a disagreement that changes sign tell you about its cause, compared with one that always leans the same way? You do not need to name the cause; Section 6.4 does that.
At 2 bits the measured worst error and half a gap are identical, at 0.61328125. The measured figure is real, from the same file; the half a gap is computed from the same real $M$ . Explain why the 2-bit case hits the ceiling exactly when the others do not. It may help to write out the three values that a 2-bit format can store.
The largest magnitude in that matrix is 45.47 times the typical magnitude, a ratio computed in Section 6.5 from two real figures, 1.2265625 and 0.02697754. Suppose somebody removed the single largest weight from the matrix before quantizing, so that $M$ became the second-largest magnitude instead. Describe what would happen to the gap, to the typical weight’s stored value, and to the weight that was removed. Then say why real quantizers do not do this.
A colleague proposes saving space by storing every weight at 2 bits, arguing that 2 bits still gives four levels and four is “plenty for a number between minus one and one”. Using the arithmetic in this chapter, write a short reply. Give at least one number, and say where it came from.
Chapter 7 will report that blockwise quantization at 4 bits, meaning one ruler for each block of 32 weights, gives a mean error worth 7.2% of the weights’ spread, while per-tensor quantization at the same 4 bits, meaning one ruler for the whole matrix, gives 46.6% (real, both, from err_over_sd in lab/out/we3b_quant.json). Nothing in those two schemes differs except what each group of weights uses as its $M$ . Using Formula 6.5, explain how changing only $M$ could account for a difference of that size, and state what you would want to measure before believing the explanation.