Streaming data / line buffers

You can't load a whole image into an FPGA at once — it's simply too much memory for on-chip storage, and even if it fit, waiting for the entire image to arrive before starting computation would waste huge amounts of time. Instead, you stream pixels in one at a time and keep just a small "sliding window" of the last couple of rows in a buffer, computing one output pixel per clock tick as data flows continuously through the circuit.

This is the core trick behind every CNN layer on FPGA. For a 3×3 convolution on a 512-pixel-wide image, you only ever need to store 2 complete rows plus the current partial row — the current 3×3 window is always available once the buffer is filled, letting you produce one output pixel every single clock cycle without ever holding the full image in memory.

The hardware implementation of this "remember the last couple of rows" idea is an SRL (Shift Register LUT) — a Xilinx-specific, highly efficient way to build a variable-length shift register using the FPGA's LUT fabric directly, without consuming a dedicated flip-flop per bit of delay. This is the hardware basis of the feed-forward line buffer architecture that a streaming CNN-on-FPGA design typically calls for.

The hands-on project here — a 3×3 Sobel edge filter using a line buffer — is structurally identical to one convolutional layer, but with simpler, known weights, so mistakes in border handling, valid-signal timing, and output timing are easy to spot before you add the complexity of real trained CNN weights on top. Border pixels are a genuinely tricky detail worth respecting: at the edges of the image, the 3×3 window doesn't have a full neighborhood, and deciding how to handle that (skip, zero-pad, replicate) is a design decision you need to make deliberately, not accidentally.