Translation-equivariant linear maps are convolutions

ProofIntermediate⏱ 30 minDeep Learningconvolutionequivariancecharacterisationweight sharing

Theory: Convolution, Recurrence and Normalisation →

Problem

Prove that a linear map $A$ on signals over $\Z_n$ commutes with all translations if and only if it is a convolution. Deduce that the weight sharing in a CNN is forced, not chosen.

Write a complete argument. Every inequality you use should be one you could state precisely.

Solution

Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.

Related problems

All problems

Problem	Field	Type	Difficulty	Time
First-order characterisation of convexity	Optimization	Proof	Beginner	30 min
Receptive field growth with depth and dilation	Deep Learning	Numerical	Beginner	15 min
Convolution diagonalises in the Fourier basis	Deep Learning	Proof	Intermediate	30 min
The RNN gradient is a repeated matrix product	Deep Learning	Proof	Intermediate	30 min
Normalisation makes weights scale-invariant	Deep Learning	Proof	Advanced	30 min