Machine Learning Foundations · Quiz 1 · Jan 2025

**1. Evaluate the tangent data:** For \(f(x,y)=\sin(xy)\), \(\nabla f=(y\cos(xy),x\cos(xy))\). At \((\pi/2,0)\), the value is 0 and the gradient is \((0,\pi/2)\). **2. Use the displacement:** The target point differs by \((0.1,0.1)\). Thus \(L=0+0(0.1)+(\pi/2)(0.1)=0.05\pi\). With the requested \(\pi\approx3.14\), this is 0.157. **Answer: C — 0.157.** A: **Incorrect:** It does not equal the first-order value 0.05π. B: **Incorrect:** It adds an unsupported increment to the tangent estimate. C: **Correct:** It equals 0.05×3.14. D: **Incorrect:** It omits the factor 0.1 from the y-displacement.

**1. Determine rank and eigenvalues:** In \(M=\begin{pmatrix}1&2\\3&6\end{pmatrix}\), the second row is three times the first, and M is nonzero. Its rank is one. The characteristic polynomial is \(\lambda^2-7\lambda=\lambda(\lambda-7)\), giving eigenvalues 0 and 7. **2. Check the other claims:** Distinct eigenvalues make M diagonalizable. Eigenvectors \((-2,1)^T\) for 0 and \((1,3)^T\) for 7 have dot product 1, so they are not orthogonal. **Answer: D — The rank of M is 1..** A: **Incorrect:** The displayed eigenvectors have nonzero dot product. B: **Incorrect:** Two distinct eigenvalues provide a basis of eigenvectors. C: **Incorrect:** The eigenvalues are 0 and 7, not 4 and 1. D: **Correct:** There is one independent nonzero row.

**1. Recall the definition:** At a differentiable point and for unit u, \[D_uf(x_0,y_0)=\lim_{h\to0}\frac{f((x_0,y_0)+hu)-f(x_0,y_0)}h=\nabla f(x_0,y_0)\cdot u.\] **2. Interpret the quantity:** This scalar measures the local rate of change per unit travel in the specified direction. The gradient norm is only the maximum directional rate, obtained when u aligns with the gradient. **Answer: A — the rate of change of f in direction u at the point.** A: **Correct:** It is exactly the local rate in the specified unit direction. B: **Incorrect:** The derivative equals the gradient norm only in a maximizing direction, not generally. C: **Incorrect:** A fixed-direction derivative is not an optimization over directions. D: **Incorrect:** The angle contributes through a cosine but is not itself the derivative.

**1. First solve the unrestricted quadratic fit:** For samples \((1,-1),(0,1),(-1,1)\), a quadratic \(q=\theta_0+\theta_1x+\theta_2x^2\) interpolates exactly with \(\theta_0=1\), \(\theta_1=-1\), \(\theta_2=-1\). Thus the true least-squares minimizer is \(q(x)=1-x-x^2\), with zero loss; it is absent from the choices. **2. Compare the actual choices:** At x=1,0,-1, A predicts 4,2,5, giving SSE 42. B predicts -1,2,0, giving SSE 2. C predicts 0,2,-1, giving SSE 6. D predicts 5,2,4, giving SSE 46. **3. Distinguish the two readings:** B has the smallest error among the four offered polynomials. It is the intended selectable answer, but it is not the unrestricted best-fitting quadratic. **Answer: B — B — best among the listed choices only.** **Convention note:** The data and options are inconsistent with the phrase “best second-order polynomial” over all coefficients. The genuine zero-loss fit is 1-x-x²; B is returned only as the best listed candidate. A: **Incorrect:** Its squared-error sum is 42. B: **Correct:** Its squared-error sum is 2, smallest among the offered choices. C: **Incorrect:** Its squared-error sum is 6, larger than B’s. D: **Incorrect:** Its squared-error sum is 46.

**1. Check the explicit dot products and norm:** \((0,0,1)\) has dot product zero with both \((1,0,0)\) and \((1,1,0)\). Also \((1,2,3)\cdot(1,1,-1)=1+2-3=0\). But \(\|(1,1,-1)\|=\sqrt3\), not 3. **2. Correct the fundamental-space relation:** The row space is perpendicular to the nullspace, while the column space is perpendicular to the left nullspace. Column and right-null spaces may even have different ambient dimensions. For a square counterexample, \(A=\begin{pmatrix}1&1\\0&0\end{pmatrix}\) has column vector \((1,0)\) and null vector \((1,-1)\) with nonzero dot product. **Answer: A, D — A and D.** A: **Correct:** Both specified dot products are zero. B: **Incorrect:** It confuses column space with row space in the nullspace orthogonality relation. C: **Incorrect:** It gives the squared norm instead of the norm. D: **Correct:** The two displayed vectors have zero dot product.

**1. Build the orthogonal projector:** The target direction is \(v=(1,-1,0)^T\), with \(v^Tv=2\). Hence \[P=\frac{vv^T}{2}=\frac12\begin{pmatrix}1&-1&0\\-1&1&0\\0&0&0\end{pmatrix}.\] **2. Project the supplied vector:** For \(x=(1,2,1)^T\), \(v^Tx=-1\), so \(Px=v(-1)/2=(-1,1,0)^T/2\). **3. Check the result geometrically:** It is parallel to v, and the residual \(x-Px=(3/2,3/2,1)^T\) is perpendicular to v. **Answer: B, C — B and C.** A: **Incorrect:** Its signs do not produce the target orthogonal projector. B: **Correct:** It is vvᵀ/(vᵀv). C: **Correct:** It uses the correct coefficient -1/2. D: **Incorrect:** It has the opposite sign from the actual projection.

**1. Check domains and derivatives:** The real logarithm is defined only for x>0, so the claim including x=0 is false. The exponential has derivative eˣ everywhere on the real line, proving B. **2. Examine the remaining functions:** The square root is continuous at zero from its domain, but \((\sqrt h-0)/h=1/\sqrt h\to+\infty\), so it has no finite derivative there. The reciprocal has derivative \(-1/x^2\) at every nonzero x and is continuous there. **Answer: B, C, D — B, C and D.** A: **Incorrect:** ln(x) is not defined at zero, which the stated interval includes. B: **Correct:** The exponential is smooth for all real x. C: **Correct:** The square root is continuous at zero but has no finite derivative there. D: **Correct:** The reciprocal is smooth on each side of its excluded point zero.

**1. Recognize rank one:** For \(A=\begin{pmatrix}2&5\\6&15\end{pmatrix}\), row 2 is three times row 1. The columns are multiples of \((1,3)^T\), giving the claimed column space. **2. Solve both nullspaces:** \(2x+5y=0\) gives \(\ker A=\operatorname{span}\{(-5,2)^T\}\). The equation \(A^T(u,v)^T=0\) gives \(u+3v=0\), so the left nullspace is spanned by \((-3,1)^T\). **3. Check the row-space sign:** The row space is \(\operatorname{span}\{(2,5)\}\), not \(\operatorname{span}\{(2,-5)\}\). A global scalar cannot change just one coordinate’s sign. **Answer: A, B, C — A, B and C.** A: **Correct:** Its generator solves the single independent row equation. B: **Correct:** Both columns are multiples of (1,3). C: **Correct:** Its generator solves the transpose-nullspace equation. D: **Incorrect:** The second coordinate should have the same sign as in (2,5).

**1. Use rank–nullity:** A \(20\times25\) matrix acts on a 25-dimensional domain. Thus \(\operatorname{rank}(A)+\operatorname{nullity}(A)=25\). **2. Substitute and check feasibility:** The rank is \(25-10=15\). This is consistent with the upper bound \(\min(20,25)=20\). Subtracting from 20 would incorrectly use the codomain dimension. **Answer: 15.**

**1. Differentiate all three terms:** For \(f(x,y,z)=\sin(xy)+e^{yz}+xz^2\), \[\nabla f=\left(y\cos(xy)+z^2,\ x\cos(xy)+ze^{yz},\ ye^{yz}+2xz\right).\] **2. Evaluate at the point:** At (1,0,2), \(xy=yz=0\), so \(\cos0=e^0=1\). Thus \(\nabla f(1,0,2)=(4,3,4)\). **3. Compute its Euclidean length:** \(\|\nabla f\|=\sqrt{4^2+3^2+4^2}=\sqrt{41}=6.403124\ldots\), which rounds to 6.40. **Answer: 6.40.**

**1. Calculate the score:** With x=(0.9,5,1), \(z=0.5(0.9)+0.3(5)-0.2(1)=0.45+1.50-0.20=1.75\). **2. Apply the threshold:** The classifier returns 1 for z≥1.5, so \(u(z)=1\). The score 1.75 is an intermediate value, not the binary prediction; the observed label 0 does not change the model’s prediction. **Answer: B — 1.** A: **Incorrect:** The score is above the threshold, not below it. B: **Correct:** The step function outputs 1 because 1.75≥1.5. C: **Incorrect:** It is the pre-threshold score z, not u(z). D: **Incorrect:** Both the score formula and threshold are supplied.

**1. Score all five training samples:** In the listed order, the scores are \(1.75,0.10,1.20,-0.70,2.15\), while the true labels are \(0,1,0,1,0\). **2. Evaluate the original threshold:** At 1.5, predictions are \(1,0,0,0,1\). Four of five labels are wrong, so the rate is 4/5=0.8. **3. Evaluate the new threshold:** At 1.0, predictions become \(1,0,1,0,1\). The third sample changes from correct to incorrect; the others are unchanged. All five are wrong, so the rate rises to 1.0. **Answer: A — Increase.** A: **Correct:** The error rate increases from 80% to 100%. B: **Incorrect:** The only changed prediction introduces an additional error. C: **Incorrect:** The third prediction changes its correctness. D: **Incorrect:** The full labeled dataset and both thresholds are available.

**1. Distribute the leading minus sign:** The matrix is \(B=\begin{pmatrix}4&-3&0\\2&-1&0\\1&-1&1\end{pmatrix}\). Its characteristic polynomial is \((\lambda-2)(\lambda-1)^2\). **2. Construct an eigenbasis:** \(v_1=(3,2,1)^T\) satisfies \(Bv_1=2v_1\). The independent vectors \(v_2=(1,1,0)^T\) and \(v_3=(0,0,1)^T\) satisfy \(Bv_j=v_j\). **3. Place them as columns:** \[P=\begin{pmatrix}3&1&0\\2&1&0\\1&0&1\end{pmatrix},\qquad\det P=1.\] It is invertible and obeys \(BP=P\operatorname{diag}(2,1,1)\), so D is valid. **Answer: D — D — P=[[3,1,0],[2,1,0],[1,0,1]].** A: **Incorrect:** Its columns include e₁, which is not an eigenvector of B. B: **Incorrect:** Its second column (2,1,0) maps to (5,3,1), not a scalar multiple of itself. C: **Incorrect:** Its first column (1,2,0) maps to (-2,-1,-1), not a scalar multiple. D: **Correct:** Its columns are the independent eigenvectors with eigenvalues 2,1,1. E: **Incorrect:** The explicit eigenbasis proves diagonalizability.

**1. Retain the selected column order:** For \(P=\begin{pmatrix}3&1&0\\2&1&0\\1&0&1\end{pmatrix}\), its columns have eigenvalues 2, 1 and 1 in that order. **2. Match eigenvalues to diagonal entries:** Therefore \(D=\operatorname{diag}(2,1,1)\), and \(BP=PD\) proves \(B=PDP^{-1}\). Its trace 4 and determinant 2 match those of B. **Answer: A — D=diag(2,1,1).** A: **Correct:** It places 2,1,1 in the order of the chosen eigenvectors. B: **Incorrect:** It contains eigenvalues 4 and -1 that B does not have. C: **Incorrect:** It incorrectly uses 3 and 2 instead of 2 and 1 for the first two columns. D: **Incorrect:** It introduces a zero eigenvalue even though B is invertible. E: **Incorrect:** The explicit invertible eigenvector matrix proves otherwise.