Any input is a sum of scaled, shifted impulses. Shift invariance says each produces the same impulse response, moved; homogeneity says it is scaled by that sample’s value; additivity says the total is their sum. Convolution is not a definition anyone chose — it falls out of those three properties.
Convolution as stamping
Interactive
Step 3: input[2] = 0.5stamps [0.165, 0.17, 0.165]into positions 2–45 samples still to stamp
Step through it. The highlighted input sample lays down its own scaled copy of the kernel — the red row in the middle block, numbered to match the sample that made it. Rows above it are stamps already placed; dashed rows are still to come. At the bottom, red shows exactly what this step added, growing out of the dashed ghost that marks the running total beforehand, while cyan marks positions this step did not touch.
All positive, summing to one — a moving average. Each stamp is a small copy of the input sample spread over three positions. Notice that nothing slides or flips — each sample simply drops a copy where it sits, and the copies pile up. The output runs to 8 + 3 − 1 = 10 samples, because the last stamp still extends past the end of the input. That overhang is where edge effects and block-overlap problems come from.
The stamping picture makes the output length obvious. An 81-sample input with a 31-point kernel gives 81 + 31 − 1 = 111 samples. Convolution always makes signals longer, and that tail is where edge effects and block-overlap problems come from.
Reading a kernel at a glance
Two rules classify most kernels immediately. All values positive and summing to one: a smoother, which passes a constant unchanged. Mixed signs summing to zero: a differencer, which rejects a constant exactly and responds only to change.
A single spike is the identity. Scale it and you have gain; shift it and you have delay; add a smaller delayed spike and you have an echo. Record the impulse response of a real concert hall and convolving with it puts any dry recording in that hall — the impulse response is a complete description of the room, exactly as the theory promises.
Four impulse responses built from nothing but delta functions
Figure
A single spike of height one is the identity — convolving with it changes nothing. Scale it and you have gain; shift it and you have a pure delay; add a smaller delayed copy and you have an echo. Nothing here is a filter in the usual sense, yet all four are legitimate linear systems, completely described by these few numbers. Extend the fourth idea to hundreds of irregularly spaced, decaying spikes and you have the impulse response of a room — which is exactly how convolution reverb works.
Zero, linear, and nonlinear phase — read off the impulse response
Figure
You can classify a filter’s phase behaviour by eye, without computing anything. Zero phase is symmetric about sample zero — no delay, no distortion, and impossible in real time because it needs future samples. Linear phase is the same shape shifted later: every frequency delayed equally, so the waveform arrives intact, and it is what a symmetric FIR kernel gives you. Nonlinear phase is symmetric about nothing — different frequencies are delayed by different amounts and the waveform is reshaped even if no frequency is attenuated. Recursive filters are always in this third category, which is why filtfilt exists.
Correlation, and the cost that motivates everything after
Correlation is convolution without the flip. Convolution asks what a system does to a signal; correlation asks how much a signal resembles a pattern at each alignment. Correlating against a copy of a known signal is a matched filter, and it maximises signal-to-noise ratio for detecting that signal in white noise — which is how radar pulse compression, modem timing recovery, and GPS acquisition below the noise floor all work.
Direct convolution costs output length times kernel length. A million-sample signal with a thousand-point kernel is a billion operations. That wall is why the next four episodes exist: convolution in the time domain equals multiplication in the frequency domain.
Key points
The delta function is a single spike of height one at position zero and zero everywhere else. The system's response to it is the impulse response, written h of n, also called the filter kernel.
The impulse response completely characterizes a linear time invariant system. It follows from decomposing the input into scaled shifted deltas and applying homogeneity, shift invariance, and additivity.
Convolution is not a definition somebody chose. It is what LTI systems inescapably do.
The clearest picture is stamping: each input sample drops a copy of the impulse response into the output, scaled by that sample's value and positioned at that sample's location. The output is the accumulation of all those overlapping copies.
Output length equals input length plus kernel length minus one. Eighty one samples convolved with a thirty one point kernel gives one hundred and eleven. Convolution always makes signals longer, which is where edge effects and block overlap problems come from.
The output side algorithm asks instead which inputs reached each output. It produces the familiar formula with x of i minus j, and the apparent "flip" of the kernel is just the bookkeeping consequence of nearby inputs contributing early kernel values and distant inputs contributing late ones.
A causal impulse response is zero for negative positions and is mandatory for real time work. Non-causal symmetric kernels give linear phase, meaning every frequency is delayed equally and the waveform shape is undistorted. Shifting a symmetric kernel later makes it causal at the cost of a delay of half its length, which is why sharp linear phase filters always add latency.
Convolution needs samples beyond both ends of the signal. You may assume zeros, extend the edge values, reflect, or wrap around. Wraparound is what the discrete Fourier transform implicitly assumes, producing circular convolution, and avoiding it is why FFT convolution needs careful zero padding. Roughly half a kernel length at each end is untrustworthy regardless of choice.
Convolving with a delta function is the identity. Scaling the delta gives gain; shifting it gives delay; a delta plus a smaller delayed delta gives an echo. Convolving with a recorded room impulse response reproduces that room's acoustics exactly.
A kernel of all ones is a running sum, the discrete integral. A kernel of plus one then minus one is a first difference, the discrete derivative, and the basis of edge detection.
Low pass kernels have all positive values summing to one, which leaves a constant unchanged. High pass kernels have mixed signs summing to zero, which rejects a constant exactly. You can classify a kernel at a glance with those two rules.
Convolution is commutative, associative, and distributive. Associativity means a cascade of filters can be pre-combined into one kernel and applied in a single pass. Distributivity means parallel filters can be combined by adding their kernels.
Correlation is convolution without the flip. It measures resemblance at each alignment rather than describing what a system does.
Correlating against a copy of a known signal is a matched filter, and it maximizes signal to noise ratio for detecting that signal in white noise. It explains radar pulse compression, modem timing recovery, and G P S acquisition below the noise floor.
Normalized correlation divides out the energy of both sequences so the result measures shape similarity rather than raw energy. For symmetric kernels, correlation and convolution are identical, which is why the distinction is easy to miss.
Direct convolution costs the number of output samples times the kernel length. A million sample signal with a thousand point kernel is a billion operations, which is the wall that motivates FFT convolution.
Convolution in the time domain equals multiplication in the frequency domain. That single fact is the most valuable property of the Fourier transform, and it is what the next four episodes build toward.