FPGAs are bad at decimals. You fake them with integers and just remember where the point is.
Floating-point math is expensive in hardware — it needs extra circuitry to separately track an exponent and a mantissa, and to handle all the edge cases that come with that (very large numbers, very small numbers, rounding). So instead, FPGA designs commonly fake decimal points using plain integers and just remember, by convention, where the decimal point sits. This is called "fixed-point," and it's exactly what quantizing your CNN to INT8 means in practice, viewed from the hardware side.
In Q-format notation, Q8.8 means 8 bits for the whole number part and 8 bits for the fraction part — 16 bits total. Nothing in the hardware "knows" there's a decimal point; the position is purely a convention you and your Verilog agree on when interpreting the raw bits.
Here's the part that causes silent bugs: when you multiply two Q8.8 numbers, the mathematically correct result needs more bits — specifically Q16.16 — and must be right-shifted and truncated back down to Q8.8 to be stored compactly again. Get the shift amount wrong, even by a single bit, and every layer gains a small but consistent bias that compounds as it flows deeper into the network. This is not a crash-level bug; it's the kind that shows up as "my FPGA accuracy is mysteriously a bit worse than my Python quantized model, across every single test case."
There's also accumulator bit-growth to account for: summing 9 products for a 3×3 convolution needs a few extra "guard bits" beyond the raw product width, or the sum silently overflows and wraps around — which looks like random noise scattered through your output, not an obvious crash or error flag. As a concrete rule of thumb: accumulating N products of B-bit values needs roughly log2(N) extra guard bits on top of the product width. For a 3×3 kernel (9 INT8×INT8 products, each 16 bits), the accumulator needs at least 4 guard bits on top of that 16-bit product width to be safe.
This phase is the direct hardware counterpart of the accuracy work you'll do on the CNN training side. Post-training quantization (PTQ) in PyTorch tells you what precision is safe to lose in software terms; this phase tells you how that same precision actually gets implemented, bit by bit, in real accumulator hardware. The two have to agree, or your "golden reference" and your FPGA output will diverge for reasons that have nothing to do with your algorithm being wrong.
This phase is the direct hardware counterpart of the CNN training-side quantization work. PyTorch's PTQ tells you what precision is safe to lose; this phase tells you how that precision actually gets implemented, bit by bit, in accumulator hardware. They have to agree, or your FPGA output diverges from your golden reference for reasons unrelated to your actual algorithm.