Receptive field growth with depth and dilation

NumericalBeginner⏱ 15 minDeep Learningreceptive fielddilationstridecnn

Theory: Convolution, Recurrence and Normalisation →

Problem

Compute the receptive field of (a) $L$ stacked $3\times3$ convolutions with stride $1$ ; (b) the same with dilation doubling each layer; (c) $L$ layers with stride $2$ . Evaluate each at $L = 10$ and comment on the trade-offs.

Compute the requested quantities and say what they mean. A number without an interpretation is not an answer.

Solution

Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.

Related problems

All problems

Problem	Field	Type	Difficulty	Time
Convolution diagonalises in the Fourier basis	Deep Learning	Proof	Intermediate	30 min
The RNN gradient is a repeated matrix product	Deep Learning	Proof	Intermediate	30 min
Translation-equivariant linear maps are convolutions	Deep Learning	Proof	Intermediate	30 min
Normalisation makes weights scale-invariant	Deep Learning	Proof	Advanced	30 min
The LSTM cell-state Jacobian	Deep Learning	Proof	Advanced	30 min