Deep Learning · End Term · Jan 2026 · FN

With stride 1, output size is \(L+2P-F+1\). Set it equal to L: \(L+2P-F+1=L\), so \(P=(F-1)/2\) on each side. Equal integer padding on both sides therefore gives exact same-size output for odd F. For even F, a same-padding implementation generally uses asymmetric integer padding rather than a fractional number of pixels. **Answer: B.** A: **Incorrect:** Would make the output two pixels larger than intended. B: **Correct:** Solves the stride-1 output-size equation. C: **Incorrect:** Gives a total padding one pixel too large for the equation. D: **Incorrect:** Overpads relative to the kernel's F-1 total padding requirement.

For input size d and hidden size h, a vanilla recurrent layer has \(hd+h^2+h\) parameters. A standard LSTM has four corresponding affine transforms (input, forget, output and candidate), giving \(4(hd+h^2+h)\), before any common output layer. Thus the vanilla RNN does not have more parameters: Statement 1 is false. Statement 2 correctly explains why the LSTM has more. **Answer: C.** A: **Incorrect:** Statement 1 reverses the actual parameter comparison. B: **Incorrect:** Both truth labels are wrong. C: **Correct:** Correctly marks Statement 1 false and Statement 2 true. D: **Incorrect:** The multiple-gate explanation in Statement 2 is true.

Compare derivatives at x=1. \(d\log_{10}x/dx=1/(x\ln10)\), giving about 0.4343. \(d\ln(x^2)/dx=2/x\), giving 2. \(d\sqrt{x}/dx=1/(2\sqrt{x})\), giving 0.5. \(d(x^3)/dx=3x^2\), giving 3. The largest slope is therefore that of x cubed. **Answer: D.** A: **Incorrect:** Its slope at 1 is approximately 0.4343. B: **Incorrect:** Its slope at 1 is 2, below 3. C: **Incorrect:** Its slope at 1 is 0.5. D: **Correct:** Its slope at 1 is 3, the largest of the four.

The hidden preactivations are \(x_1-x_2\) and \(x_2-x_1\). ReLU keeps only their positive parts, and the output sums them: \(\hat y=\max(0,x_1-x_2)+\max(0,x_2-x_1)=|x_1-x_2|\). It gives 0 on (0,0),(1,1) and 1 on (1,0),(0,1). A threshold at 0.5 therefore separates the two XOR classes exactly. **Answer: A.** A: **Correct:** The computed values 0 and 1 lie on opposite sides of 0.5 for the required classes. B: **Incorrect:** The hidden nonlinear transformation makes the classes separable, even though one linear input boundary cannot.

Encoder self-attention allows source tokens to attend to source tokens: P–3. Masked decoder self-attention blocks future target tokens: Q–2. Decoder cross-attention uses decoder queries with encoder keys and values: R–1. The feed-forward sublayer acts independently at each position after attention: S–4. Distinguish which sequence supplies queries and whether future positions are visible. **Answer: C.** A: **Incorrect:** Swaps the encoder and masked-decoder roles. B: **Incorrect:** Swaps masked self-attention with decoder-to-encoder cross-attention. C: **Correct:** Matches P–3, Q–2, R–1, S–4. D: **Incorrect:** Misassigns the positionwise feed-forward and encoder-attention roles.

Every layer's affine preactivation is zero because all weights are zero and biases are ignored. The sigmoid is \(\sigma(z)=1/(1+e^{-z})\), so each hidden unit outputs sigma(0)=0.5. The output layer still has zero preactivation because its own weights are zero, and its sigmoid therefore outputs 0.5. The input values and depth do not change this initial forward result. **Answer: 0.5.**

Hidden activations remain 0.5, but the output weights are all zero and there is no bias. Thus the output preactivation is a sum of zero-weighted hidden activations, equal to zero. A linear output returns that value unchanged, so the answer is 0 rather than the sigmoid output 0.5. **Answer: 0.**

The input has one channel and the convolution uses one 3×3 filter with no bias. Its parameter count is \(3\cdot3\cdot1\cdot1=9\). ReLU and max pooling have no learned coefficients. The filter is reused at every spatial position, so output size does not multiply the number of trainable parameters. **Answer: 9.**

Using the CNN's cross-correlation convention, the convolution map is \(\begin{bmatrix}3&9&1\\-3&-1&3\\-3&-5&-7\end{bmatrix}\). ReLU replaces negatives with zero. The 2×2 stride-2 pool takes the upper-left 2×2 block, whose unique maximum is 9 at convolution position (row 1,column 2) in one-based indexing. Thus yhat=9. For the selected convolution output, W12 multiplies the image entry at row 1,column 3, which is 0. Consequently \(\partial L/\partial W_{12}=(9-1)\cdot1\cdot1\cdot0=0\). The zero comes from the selected input coefficient, not from a zero residual. **Answer: 0.**

Training error 10% versus test error 56% suggests poor generalisation, consistent with overfitting. Input noise can encourage robustness and L2 regularisation can discourage excessively large weights. These can help, though neither guarantees a particular test error. Switching optimiser or activation alone does not specifically constrain overfitting. Evaluation should also check that training and test distributions and labels are comparable. **Answer: C, E.** A: **Incorrect:** Adam changes optimisation dynamics but is not a direct regularisation remedy by itself. B: **Incorrect:** ReLU is an activation choice, not a direct response to this generalisation gap. C: **Correct:** Input noise can act as regularisation and improve robustness. D: **Incorrect:** Maxout changes the activation/model capacity and is not the targeted regularisation method here. E: **Correct:** An L2 penalty can reduce overfitting by constraining weight magnitudes.

A query-key dot product sums dk component products; its variance can grow with dk. Dividing by \(\sqrt{d_k}\) controls scale before softmax and helps avoid saturation. Standard heads have separate learned projections. If every head has identical Q,K,V projections and scaling, each computes the same attention distribution and output, so repetition supplies no distinct attention pattern; their concatenation followed by a linear output projection can be represented by an effective single such head. At fixed model dimension, splitting projections across more narrower heads need not increase total Q,K,V parameters. **Answer: A, C.** A: **Correct:** The square-root scale counteracts growth in dot-product magnitude with key dimension. B: **Incorrect:** Standard multi-head attention learns separate projections for its heads. C: **Correct:** Identical heads are redundant under the stated identical-projection/scaling interpretation; the output projection can combine their repeated output. D: **Incorrect:** With fixed total projection dimensions, the number of heads alone need not change total projection weights.

Remove the specified stop words before forming adjacent contexts. The sequences become [students,read,books,library], [teachers,read,books,after,class], and [students,teachers,discuss,library]. In the first two sentences, books is immediately to the right of read, contributing one occurrence each to row read, column books. The third sentence contains neither read nor books. Therefore X[read,books]=2. **Answer: 2.**

The filtered sentences have lengths 4,5,4. With one context word on each side, their total directed context pairs number \(2(3+4+3)=20\). The read-books pair occurs twice, and read and books each have marginal context counts 4. Thus \[PMI(read,books)=\ln\frac{2/20}{(4/20)(4/20)}=\ln(2.5)>0.\] They co-occur more often than this marginal-independence model predicts. **Answer: A.** A: **Correct:** The observed joint probability 0.1 exceeds the marginal product 0.04. B: **Incorrect:** Zero would require equality between the joint probability and marginal product. C: **Incorrect:** Negative PMI would require the joint probability to be smaller than the marginal product.

For one head, attention weights have shape t×t. Multiplying them by the value vectors produces one dv-dimensional result per token, hence output shape t×dv (or its transpose under column-vector notation). With t=32 and dv=16, the number of elements is \(32\cdot16=512\). Do not multiply by the number of heads because the question asks for a single head. **Answer: 512.**

Here dmodel=64, h=2, and dk=dq=64/2=32, while dv=16. Each head has Q and K projections of size 32×64 and a V projection of size 16×64. Across two heads this gives \(2(2048+2048+1024)=10240\). The output projection has shape 64×(2×16), adding 2048. Total=10240+2048=12288. Biases are excluded, and sequence length affects computation rather than these shared parameters. **Answer: 12288.**

The first FFN map goes from 64 to 256 dimensions, so it has \(64\cdot256=16384\) weights and 256 biases. The second returns from 256 to 64, adding 16384 weights and 64 biases. ReLU adds none. Thus total \(=16384+256+16384+64=33088\). These same maps are applied at every token; do not multiply by sequence length. **Answer: 33088.**

For a one-hot target, cross entropy is \(-\sum_i y_i\ln p_i\). Every target entry is zero except the correct word, so the loss reduces to \(-\ln(0.2)=\ln5\approx1.609438\). Rounded to two decimals, it is 1.61. The vocabulary size and the target's index identify the classification task but do not introduce additional terms in this one-hot loss. **Answer: 1.61.**

There are nine distinct words. With two context positions on each side, the context counts from the first to last target are 2,3,4,4,4,4,4,3,2. Their sum is 30. Equivalently there are 8 adjacent unordered position pairs and 7 distance-two pairs, each producing both target-context directions: 2×(8+7)=30. Since every word is distinct, no two ordered word pairs collapse as duplicates. Contexts do not extend beyond sentence boundaries. **Answer: 30.**