Linear Systems and Superposition

The two conditions

Hand someone an arbitrary rule mapping input signals to output signals and there is essentially nothing useful they can say about it. Impose two conditions and an enormous amount becomes knowable — you can characterise the whole system with a single measurement and predict its response to any input whatsoever.

Homogeneity: scaling the input scales the output identically. An amplifier driven into saturation fails this — double the input and the output does not double, because it hits the ceiling.

Additivity: summed inputs produce summed outputs, with no interaction. A squarer fails this: feed in a plus b and you get an extra cross term that was in neither output alone. In radio that cross term is intermodulation, and it is exactly why nonlinearity in a receiver is such a problem.

A third property, shift invariance, is usually required alongside them: delay the input and the output delays identically. Linearity plus shift invariance is what people almost always mean by “linear system”.

Sinusoidal fidelity

Here is the consequence that organises the rest of the series. Put a sine wave into a linear system and the output is a sine wave at exactly the same frequency. The amplitude may change and the phase may change. The frequency cannot.

Put negatively: a linear system cannot create a frequency, cannot shift one, and cannot generate a harmonic. Whatever goes in comes out — scaled and phase shifted, never moved.

This is why the Fourier transform matters. Not because decomposing signals into sine waves is inherently interesting, but because sinusoids are the signals that pass through linear systems keeping their identity. They are the natural coordinate system for the problem.

Superposition, and choosing your decomposition

The strategy: break a complicated signal into a sum of simple ones, work out what the system does to each simple piece, and add the individual outputs. Because the system is linear, that sum is exactly right, not approximately.

The freedom people underuse is that you choose the decomposition. Break a signal into individual samples and one measurement — the impulse response — determines the response to every possible input, because shift invariance handles position and homogeneity handles scale.

Other decompositions lead elsewhere. Steps give you the step response, which is the natural language for rise time and overshoot. Splitting into even and odd samples is the entire basis of the FFT. And decomposing into sinusoids is the Fourier transform, which dominates because of sinusoidal fidelity above.

Key points

1. A system takes an input signal and produces an output signal. Without constraints, essentially nothing general can be said about one.
2. Linearity requires homogeneity — scaling the input scales the output identically — and additivity, meaning summed inputs produce summed outputs with no interaction between them.
3. Shift invariance is a separate property: delaying the input delays the output identically. Linearity plus shift invariance gives LTI, which is what "linear system" usually means in practice. Adaptive filters are linear but deliberately not shift invariant.
4. Sinusoidal fidelity is the crucial consequence. A sine into a linear system gives a sine out at exactly the same frequency, with only amplitude and phase changed. A linear system can never create, move, or add a frequency.
5. Because of that, a complete description of an LTI system is its frequency response — the gain and phase shift at every frequency. This is precisely why the Fourier transform is central: sinusoids are the signals that keep their identity through linear systems.
6. Frequency translation is impossible in a linear system, so any system that shifts a signal in frequency — a radio mixer, for example — must be nonlinear by design.
7. Adding a constant is not linear. Linearity requires zero in to give zero out; a straight line with an offset is affine and fails both tests. This is one reason removing the mean is such a common first step.
8. Cascaded LTI systems commute — passing through A then B gives the identical result to B then A. Mathematically exact, though in finite precision the ordering can still affect overflow and round-off.
9. Superposition is the foundation: decompose a hard signal into simple pieces, find the response to each, and add the responses. The result is exactly correct, and you may choose whichever decomposition makes your problem easiest.
10. Impulse decomposition breaks a signal into single spikes. Since one measurement — the impulse response — determines the response to every spike by scaling and shifting, it determines the response to any input at all. Adding up those scaled shifted copies is convolution.
11. Step decomposition leads to the step response, which is the natural language for time domain behavior: rise time, overshoot, settling, ringing. The impulse response is the difference of adjacent step response samples; the step response is the running sum of the impulse response.
12. Even and odd decomposition matters for Fourier symmetry properties; interlaced decomposition into even and odd numbered samples is the basis of the fast Fourier transform.
13. When a system is not linear, you can keep the signal small enough that it nearly is, linearize about an operating point for small variations, transform the problem into a domain where it becomes linear, or deliberately use nonlinear methods.
14. Homomorphic processing takes logarithms to convert multiplication into addition, applies linear methods there, and transforms back. Separating vocal cord excitation from vocal tract filtering works this way, in a domain called the cepstrum.
15. The median filter removes impulsive noise and preserves edges in a way no linear filter can, because discarding an outlier requires a rank ordering. Linearity's guarantees come at the cost of permanently excluding anything that requires a decision or a threshold.