Introduction to Deep Learning and Generative AI · Quiz 1 · Jan 2026

**1. Count tensor elements:** x has \(4\times3\times2=24\) elements, y has \(6\times4=24\), and z has 5. **2. Evaluate the expression:** numel counts entries, independently of their random values. Thus \(24+24-5=43\). **Answer: B — 43.** A: **Incorrect:** It does not equal the element-count expression. B: **Correct:** 24+24−5 equals 43. C: **Incorrect:** 48 omits the subtraction of z.numel(). D: **Incorrect:** 53 adds the final 5 instead of subtracting it.

**1. Apply squeeze(0):** x has shape (1,5,1,3,1). Removing only the leading size-one axis gives y shape (5,1,3,1). **2. Apply squeeze():** With no dimension specified, every size-one axis is removed, giving z shape (5,3). **Answer: C — torch.Size([5,1,3,1]), torch.Size([5,3]).** A: **Incorrect:** The first shape is right, but the trailing singleton in z is also removed. B: **Incorrect:** squeeze(0) removes the leading singleton rather than keeping it. C: **Correct:** Both the targeted and all-singleton removals are correct. D: **Incorrect:** squeeze(0) does not remove the other singleton axes.

**1. Identify optimizer registration:** In the standard SGD(model.parameters(), …) setup, assigning nn.Parameter objects to a module registers them as trainable parameters. **2. Compare the definitions:** Model B wraps both W and b in nn.Parameter. Model A creates ordinary tensors with gradient tracking but does not register them; Model C neither registers parameters nor enables their gradients. **Answer: B — Model B.** **Convention note:** This assumes the standard optimizer over model.parameters(). Explicitly passing Model A’s tensors to an optimizer would also permit updates. A: **Incorrect:** requires_grad=True alone does not register an ordinary tensor in model.parameters(). B: **Correct:** nn.Parameter registers both tensors and enables gradients by default. C: **Incorrect:** The ordinary tensors are unregistered and do not require gradients.

**1. Locate the missing reset:** The loop calls backward() and optimizer.step(), but never clears previous gradients. **2. Explain the consequence and fix:** PyTorch accumulates into parameter .grad, so successive updates include earlier gradients. Call optimizer.zero_grad() before each new backward pass when accumulation is not intended. **Answer: B — Gradients accumulate across epochs, leading to incorrect updates.** A: **Incorrect:** y_pred=model_0(X_train) is the forward pass. B: **Correct:** Missing zero_grad causes unintended accumulation across epochs. C: **Incorrect:** eval changes layer behavior and is not required for optimizer updates. D: **Incorrect:** The shown differentiable loss can produce gradients.

**1. Count Model A weights:** Flattening gives 100 inputs connected to 5 outputs, with no bias: \(p_A=100(5)=500\). **2. Count Model B weights:** Each of five 10×10 filters spans the single grayscale channel: \(p_B=5(10)(10)(1)=500\). Therefore the counts are equal. **Answer: B — \(p_A=p_B=500\).** A: **Incorrect:** Both counts are 500, so A is not larger. B: **Correct:** Both architectures use 500 weights without biases. C: **Incorrect:** Both counts are 500, so A is not smaller.

**1. Read the positional arguments:** Conv2d takes in_channels, out_channels and kernel_size in that order. The required first argument is 16. **2. Select the matching layer:** Conv2d(16,8,3,stride=1,padding=0) accepts 16 input channels and produces 8 output channels. **Answer: A — nn.Conv2d(16,8,3,stride=1,padding=0).** A: **Correct:** Its first argument correctly specifies 16 input channels. B: **Incorrect:** It expects 8 input channels; 16 is its output channel count. C: **Incorrect:** It expects 8 input channels; 16 is the kernel size. D: **Incorrect:** It expects 8 input channels; 16 is the stride.

**1. Inspect the dataset constructor:** Although a transform pipeline is defined, the FashionMNIST call does not pass transform=transform. **2. Follow dataset access:** With its default transform=None, the dataset returns PIL images and integer labels; the unused pipeline does not automatically run. **Answer: D — The dataset will return PIL images and not tensors..** A: **Incorrect:** The defined transformation is never supplied to the dataset. B: **Incorrect:** No normalization runs, and grayscale alone does not imply a [−1,1] range. C: **Incorrect:** Transforms are not automatically attached to either split. D: **Correct:** The image component remains a PIL image when transform is omitted.

**1. Pool spatially:** A 2×2 pool with stride 2 maps 6×6×2 to 3×3×2, preserving channel count. **2. Apply the pointwise convolution:** With stride 1, four 1×1 filters preserve 3×3 and create four output channels, giving 3×3×4. **Answer: B — 3×3×4.** A: **Incorrect:** This is the intermediate pooled shape before convolution. B: **Correct:** Pooling gives 3×3 and the convolution gives 4 output channels. C: **Incorrect:** Pooling halves both spatial dimensions. D: **Incorrect:** The four filters produce four channels, not 2×4 channels.

**1. Preserve spatial positions:** All four paths have matching 28×28 grids, so concatenation can occur along the channel axis. **2. Add channels:** \(64+128+32+32=256\), giving 28×28×256. **Answer: C — 28×28×256.** A: **Incorrect:** Depth concatenation does not add heights. B: **Incorrect:** It does not add widths. C: **Correct:** The channel counts sum to 256 and the spatial grid is unchanged. D: **Incorrect:** Neither spatial dimension is multiplied by the branch count.

**1. Apply GAP:** Averaging the 49 spatial values separately in each channel gives a 512-dimensional vector. **2. Count classifier weights:** Ten outputs need \(512\times10=5120\) weights. Flattening would instead feed 49×512 inputs and require 250880 weights. **Answer: B — B — 512 GAP features and 5,120 classifier weights.** **Convention note:** B calls 5120 the total trainable parameters, which excludes biases. With the usual ten output biases, the layer has 5130 parameters. The question does not specify bias=False; B is its intended weights-only answer. A: **Incorrect:** 49×512 describes flattening rather than global averaging. B: **Correct:** It correctly gives 512 GAP features and 5120 classifier weights, under the intended no-bias count. C: **Incorrect:** GAP preserves channels and collapses spatial axes, giving 1×1×512. D: **Incorrect:** GAP greatly reduces the final layer’s input dimension.

**1. Stack on a new axis:** Stacking three (3,4) tensors along dim=1 inserts an axis of length 3, giving (3,3,4). **2. Squeeze:** No axis has length 1, so the shape remains (3,3,4). **3. Count storage:** There are \(3\times3\times4=36\) float16 elements at 2 bytes each, giving 72 bytes. **Answer: A — torch.Size([3,3,4]), torch.Size([3,3,4]), 72.** A: **Correct:** Both shapes and the 72-byte tensor-data count match. B: **Incorrect:** No dimension of length 1 exists to squeeze away. C: **Incorrect:** stack inserts an axis; it does not concatenate into a (9,4) tensor. D: **Incorrect:** 36 is the element count, while float16 requires 72 bytes.

**1. Use the receptive-field recurrence:** A stride-one 3×3 convolution adds two positions to each side length: \(r_{l}=r_{l-1}+2\), starting at r0=1. **2. Apply three layers:** The lengths are 3,5,7, so the final receptive field is 7×7. **Answer: C — 7×7.** A: **Incorrect:** This is the receptive field after one layer. B: **Incorrect:** This is the receptive field after two layers. C: **Correct:** After three layers the side length is 1+3×2=7. D: **Incorrect:** 9×9 would require four such layers.

**1. Impose equal input and output sizes:** At stride 1, \(n_{out}=n+2p-k+1\). Setting nout=n gives \(p=(k-1)/2\). **2. Substitute each kernel:** For 5×5, p=2 per side; for 3×3, p=1 per side. **Answer: A — Case I: 2 pixels per side; Case II: 1 pixel per side.** A: **Correct:** These per-side paddings preserve 32×32 in the respective cases. B: **Incorrect:** It reverses the required paddings. C: **Incorrect:** Zero padding would shrink the 3×3 case to 30×30. D: **Incorrect:** These are the total paddings across two sides, not the padding on each side.

**1. Count fixed filter parameters:** One 12×12 filter over four channels has \(12\times12\times4=576\) weights, plus one bias if enabled. **2. Resize only the input:** Increasing spatial size from 12×12 to 14×14 changes the number of filter placements, not the filter coefficients. Additional parameters: 0. **Answer: 0.**

**1. Apply random masking during training:** Standard dropout samples new masks independently of which units were dropped in earlier epochs; it does not schedule every neuron to be dropped in turn. **2. Apply evaluation behavior:** Standard dropout is disabled during inference, so all neurons are retained. Select the statements that contradict these rules. **Answer: A, B, C — A, B and C — the intended false statements.** **Convention note:** C literally says all neurons have “equal chances,” which is also true if all chances are zero. Its intended false claim is that random dropping continues during inference. A: **Correct:** False, so select it: earlier masks do not force C to be dropped next. B: **Correct:** False, so select it: earlier masks do not guarantee a later all-retained mask; C has not even been dropped in the observations. C: **Correct:** False in the intended sense, so select it: dropout does not randomly drop neurons in standard inference. D: **Incorrect:** True, so do not select it: evaluation disables standard dropout.

**1. Count convolution parameters:** Conv1 has \(16(3\times3\times3+1)=448\). Conv2 has \(32(16\times3\times3+1)=4640\). The +1 includes a bias for each output channel. **2. Count dense parameters:** fc1 has \((32\times8\times8+1)128=262272\). fc2 has \((128+1)10=1290\). **3. Sum:** \(448+4640+262272+1290=268650\). Max pooling contributes no parameters. **Answer: 268650.**

**1. Compute hidden linear scores:** For x=(1,−1), the scores are \(1-(-1)=2\) and \(-0.5(1)+(-1)-0.5=-2\). **2. Apply ReLU and the output layer:** Hidden activations become (2,0), so \(y=1(2)+0.5(0)+0=2\). **Answer: C — 2.** A: **Incorrect:** The first hidden neuron is active. B: **Incorrect:** The first hidden activation is 2, not 1. C: **Correct:** The linear output is 2+0=2. D: **Incorrect:** The second hidden neuron is inactive and contributes zero.

**1. Forward through the fixed network:** For x=(1,−1), hidden scores (2,−2) become (2,0) after ReLU; the output is \(y_{pred}=2\). **2. Compute single-sample MSE:** With target 1.5, \(L=(2-1.5)^2=0.25\). No 1/2 factor is specified. **Answer: 0.25.**

**1. Write the first hidden activation:** Its weights are (1,−1) and its bias is zero, hence \(h_0=\max(0,x[0]-x[1])\). **2. Solve for inactivity:** \(h_0=0\) exactly when \(x[0]-x[1]\le0\), i.e. \(x[0]\le x[1]\). **Answer: D — \(x[0]\le x[1]\).** **Convention note:** B and C each describe a sufficient subset; D is the complete necessary-and-sufficient condition intended by this single-choice question. A: **Incorrect:** A strictly greater first input activates the neuron. B: **Incorrect:** Equality is sufficient but misses all strictly smaller first inputs. C: **Incorrect:** Strict inequality misses the zero output at equality. D: **Correct:** This includes every input at which the ReLU output is zero.

**1. Compute hidden scores:** For x=(1,0), the hidden linear layer gives (2,−1), so \(h=(\sigma(2),\sigma(-1))\approx(0.880797,0.268941)\). **2. Compute output:** \[z=h_0-h_1\approx0.611856,\qquad\hat y=\sigma(z)\approx0.648364.\] Rounded to three decimals: 0.648. **Answer: C — 0.648.** A: **Incorrect:** Zero output score would be needed for a prediction of 0.5. B: **Incorrect:** 0.612 is approximately the pre-activation, before the final sigmoid. C: **Correct:** The final sigmoid gives approximately 0.648. D: **Incorrect:** This would be sigmoid(1), but the actual output score is about 0.612.

**1. Use the supplied output value:** The question explicitly gives prediction 0.334 and target 0. For sigmoid with BCE, \(\partial L/\partial z=\hat y-y=0.334\). **2. Differentiate with respect to hidden activations:** Since \(z=h_0-h_1\), \[\nabla_hL=0.334(1,-1)=(0.334,-0.334).\] No hidden sigmoid derivative is needed when differentiating with respect to h itself. **Answer: B — [0.334, −0.334].** **Convention note:** Recomputing the earlier printed network at x=(0,1) gives 0.351636, rather than the 0.334 stipulated here. Using the stipulated value gives B; using the exact forward pass gives [0.351636,−0.351636], which is not listed. A: **Incorrect:** The second output weight is negative, so the second gradient is negative. B: **Correct:** It multiplies the given output error by the output weights (1,−1). C: **Incorrect:** Both signs are reversed. D: **Incorrect:** It introduces an extra shrinking factor not needed for a gradient with respect to h.

**1. Compare hidden neurons:** Their pre-activations are \(a_0=2x_1-x_2\) and \(a_1=-x_1+2x_2\). Their difference is \(3(x_1-x_2)\), so sigmoid monotonicity gives \(h_0>h_1\) exactly when x1>x2. **2. Relate the output to hidden preference:** \(\hat y=\sigma(h_0-h_1)\). With the usual 0.5 decision threshold, class 1 corresponds to h0 being at least as large as h1; class 0 corresponds to h1>h0. **Answer: A, C, D, F — A, C, D and F.** **Convention note:** “Larger/highly activated” is interpreted relative to the other hidden neuron. It need not mean an activation close to 1: both can be very small while one is larger. At equal activations the output is exactly 0.5, so its class depends on the tie convention. A: **Correct:** x1>x2 makes neuron 0 more active than neuron 1. B: **Incorrect:** It reverses the hidden-neuron comparison. C: **Correct:** x2>x1 makes neuron 1 more active than neuron 0. D: **Correct:** Correct in the intended relative sense: class 1 is supported by neuron 0 rather than neuron 1. E: **Incorrect:** Class 0 corresponds to the other hidden neuron being more active. F: **Correct:** Correct in the intended relative sense: class 0 is supported by neuron 1.