Introduction to Deep Learning and Generative AI · Quiz 1 · Sep 2025

**1. Interpret the entry:** This is a zero-mark administrative instruction, not an academic problem. It says to assume necessary imports in the following code questions; option A is its supplied acknowledgement. **Answer: A — Instructions have been mentioned above.** **Convention note:** This entry is retained to preserve the exported scope; it has no mathematical answer to derive. A: **Correct:** This is the acknowledgement selected by the source key. B: **Incorrect:** It is not the acknowledgement selected in the source, though the entry itself has zero marks.

**1. Decode the index:** t has shape (1,3,3). Index 0 selects its only outer matrix, : retains all rows, and 1 selects the second column. **2. Read the column:** The selected values are 2,5,8; selecting a single column index removes that axis and returns a length-three tensor. **Answer: B — torch.tensor([2,5,8]).** A: **Incorrect:** This is column 0, rather than column 1. B: **Correct:** These are the three entries of the requested second column. C: **Incorrect:** This is column 2, rather than column 1. D: **Incorrect:** This is the second row, not the second column. E: **Incorrect:** This keeps only the central entry and omits two selected rows.

**1. Stack at dimension 0:** Three copies of a length-four tensor gain a leading length-three axis, giving (3,4). **2. Stack at dimension 1:** The new length-three axis follows the original length-four axis, giving (4,3). Stack creates a new axis rather than joining elements along an existing one. **Answer: B — (3,4) and (4,3).** A: **Incorrect:** The second stack inserts its new axis at dimension 1 and has shape (4,3). B: **Correct:** Both shapes follow the specified new-axis positions. C: **Incorrect:** The first stack has shape (3,4), not (4,3). D: **Incorrect:** It reverses the two results.

**1. Read the nesting:** The tensor has shape (1,1,2,2). The first two axes are singletons. **2. Remove singleton axes:** squeeze() removes both size-one axes and preserves the two size-two axes, giving shape (2,2). **Answer: B — (2,2).** A: **Incorrect:** It removes only one of the two singleton axes. B: **Correct:** Both singleton axes are removed, leaving the 2×2 matrix. C: **Incorrect:** squeeze does not remove or flatten size-two axes. D: **Incorrect:** This is the original unsqueezed shape. E: **Incorrect:** squeeze never adds a trailing singleton axis.

**1. Locate the mismatch:** The model is already on device, while the newly created input_batch remains on the CPU. **2. Move and retain the input:** Tensor.to returns the tensor on the selected device. Assign input_batch=input_batch.to(device) before the forward pass so inputs and model weights match. **Answer: D — input_batch = input_batch.to(device).** A: **Incorrect:** The model is already on the chosen device; this does not move the input. B: **Incorrect:** Tensor.cuda returns a converted tensor; discarding that return value leaves input_batch unchanged. C: **Incorrect:** Enabling gradients does not change the device. D: **Correct:** It moves the input to the model’s device and retains the returned tensor. E: **Incorrect:** Changing a default does not move the already-created input tensor and is not a targeted fix.

**1. Identify unnecessary inference work:** Ordinary gradient-tracked forward computation can record operations and intermediate values for backpropagation. **2. Disable tracking:** torch.no_grad() suppresses reverse-mode autograd recording in its scope, reducing memory and tracking overhead for prediction. It does not itself change the model’s device or train/eval mode. **Answer: C — It disables the creation of the computational graph, saving memory and .** A: **Incorrect:** It is not an optimizer lock; forward evaluation alone normally does not update weights anyway. B: **Incorrect:** The parameters remain allocated on their existing device. C: **Correct:** Suppressing the backward graph avoids unnecessary inference memory and tracking work. D: **Incorrect:** The computation can still run on a GPU.

**1. Track axis identities:** The original order is (H,W,C), or axes (0,1,2). The target order (C,H,W) therefore requires (2,0,1). **2. Reorder rather than reshape:** permute(2,0,1) moves each channel to the correct axis without mixing pixel identities. Equal height and width do not make an incorrect axis swap semantically correct. **Answer: D — img_tensor = img_tensor.permute(2,0,1).** A: **Incorrect:** view changes shape without performing the required axis permutation. B: **Incorrect:** It retains the original HWC shape and order. C: **Incorrect:** transpose(0,2) produces (C,W,H), swapping the spatial roles even though both lengths are 224. D: **Correct:** It correctly maps HWC to CHW.

**1. Count each affine layer:** A layer with a inputs and b outputs has \(ab+b\) parameters. Thus 4→6 gives 24+6=30; 6→3 gives 18+3=21; 3→2 gives 6+2=8. **2. Add:** \(30+21+8=59\). ReLU has no trainable parameters. **Answer: 59.**

**1. Identify mode-dependent layers:** Training dropout samples masks, while evaluation dropout is the identity. BatchNorm uses current batch statistics in training and running statistics in evaluation. **2. Remove both differences:** Removing both Dropout and BatchNorm leaves only fixed Linear and ReLU layers, which evaluate the same function in train and eval modes. **Answer: C — Remove both Dropout and BatchNorm.** **Convention note:** This is the structural guarantee for the given architecture. Accidental numerical equality can occur without removals, but is not guaranteed. A: **Incorrect:** BatchNorm would still use different statistics in the two modes. B: **Incorrect:** Dropout would still differ between the two modes. C: **Correct:** Removing both mode-dependent layers makes the remaining function identical. D: **Incorrect:** ReLU behaves the same in both modes and is not the cause. E: **Incorrect:** Linear layers themselves behave identically in train and eval mode.

**1. Form the score:** With zero weights and input, \(z=w^Tx+b=b\). **2. Invert the sigmoid:** \(1/(1+e^{-b})=0.5\) implies \(e^{-b}=1\), hence \(b=0=\ln1\). **Answer: C — \(\ln1=0\).** A: **Incorrect:** sigmoid(0.5) is about 0.622, not 0.5. B: **Incorrect:** sigmoid(ln5)=5/6, not 0.5. C: **Correct:** ln1=0 and sigmoid(0)=0.5. D: **Incorrect:** sigmoid(1) is about 0.731, not 0.5.

**1. Differentiate BCE with sigmoid:** For \(z=w^Tx+b\), \(\hat y=\sigma(z)\), and \(L=-y\ln\hat y-(1-y)\ln(1-\hat y)\), the chain rule simplifies to \(\partial L/\partial z=\hat y-y\). **2. Update the bias:** Since \(\partial z/\partial b=1\), \[b_{new}=b-\eta(\hat y-y).\] **Answer: D — \(b\leftarrow b-\eta(\hat y-y)\).** A: **Incorrect:** The gradient contains neither a squared prediction nor this reversed residual. B: **Incorrect:** The BCE derivative does not square the target. C: **Incorrect:** Multiplication by x gives a weight gradient, not the bias gradient. D: **Correct:** It subtracts the learning rate times the correct bias gradient.

**1. Follow the hidden paths:** Only x1 and x3 connect to the hidden neuron, giving \(h=\max(w_1x_1+w_3x_3,0)\). **2. Follow the output paths:** The output combines w4h and the direct x2 contribution w2x2, then applies ReLU: \[\hat y=\max\{w_4\max(w_1x_1+w_3x_3,0)+w_2x_2,0\}.\] **Answer: B — \(\max\{w_4\max(w_1x_1+w_3x_3,0)+w_2x_2,0\}\).** A: **Incorrect:** It routes x2 through the hidden neuron instead of its direct output connection. B: **Correct:** It matches both hidden inputs, the direct x2 connection and both ReLUs. C: **Incorrect:** It omits the hidden ReLU and incorrectly scales the x2 path by w4. D: **Incorrect:** It has the correct connections but omits the hidden ReLU.

**1. Forward pass:** With all weights 1 and x=(0,1,0), the hidden score is 0 and h=ReLU(0)=0. The output score is \(1(0)+1(1)=1\), so \(\hat y=1\). **2. Differentiate the stated half-squared loss:** \(\partial L/\partial\hat y=\hat y-y=1-0=1\). The output ReLU has derivative 1 at score 1. **3. Differentiate the direct weight:** \(\partial z_{out}/\partial w_2=x_2=1\). Therefore \(\partial L/\partial w_2=1\times1\times1=1\). The hidden ReLU’s kink does not lie on this direct path. **Answer: 1.**

**1. Recompute the required gradient:** All weights start at 1; with x=(0,1,0), y=0, the output is 1 and \(\partial L/\partial w_2=1\) under the stated half-squared loss. **2. Apply gradient descent:** The desired update satisfies \(0=w_2^{new}=1-\eta(1)\), hence \(\eta=1\). **Answer: 1.** **Convention note:** The reference to question 14 means the immediately preceding gradient subquestion in this passage, despite its different printed global number.

**1. Count one filter:** conv1 takes one input channel and uses a 3×3 kernel, so each output filter has 9 weights and one bias. **2. Multiply by output channels:** Eight filters give \(8(9+1)=80\) trainable parameters. **Answer: 80.**

**1. Inspect both layers:** pool1 and pool2 are standard MaxPool2d operations. They return local maxima using a fixed kernel size and stride, without learning weights or biases. **2. Add their parameter counts:** Each has zero trainable parameters, so the total is 0. **Answer: 0.**

**1. Read the fc1 dimensions:** Its input has \(16\times7\times7=784\) flattened features and its output has 128 neurons. **2. Count weights and biases:** \[784(128)+128=100352+128=100480.\] **Answer: 100480.**

**1. Count output filters in each convolution:** conv1 has 8 output channels, so it has 8 filters. conv2 has 16 output channels, so it has 16 filters, each spanning all 8 input channels. **2. Sum:** \(8+16=24\) convolution filters. Pooling and dense layers are not included. **Answer: 24.**

**1. Use the branch-one input size:** The preceding 1×1, stride-one, unpadded convolution leaves the spatial dimension at 28. **2. Preserve size through the 3×3 convolution:** At stride 1, \(28=28+2p-3+1\). Solving gives p=1 pixel on each side. **Answer: 1.**

**1. Read the branch-two grid:** Its 1×1, stride-one, unpadded convolution preserves the 28×28 input grid. **2. Solve for padding:** \(28=28+2p-5+1\), so 2p=4 and p=2 pixels per side. **Answer: 2.**