Machine Learning Foundations · Quiz 1 · May 2025

**1. State the projection convention:** For orthogonal projection onto a real subspace, \(P^T=P\) and \(P^2=P\). Being real symmetric guarantees orthogonal diagonalizability; its eigenvalues are 0 or 1. **2. Separate orthogonal projection from an orthogonal matrix:** An orthogonal matrix must satisfy \(P^TP=I\). A proper subspace projector such as \(\operatorname{diag}(1,0)\) instead satisfies \(P^TP=P\ne I\). Its rank equals the target dimension, which can be 0, 1, 2 or higher. **Answer: A, C — A and C.** **Convention note:** The question uses “projection matrix” to mean an orthogonal projector. An arbitrary oblique projector need only be idempotent and need not be symmetric. A: **Correct:** An orthogonal projector is symmetric. B: **Incorrect:** A projection onto a proper subspace is singular, so it is not an orthogonal matrix. C: **Correct:** The real symmetric spectral theorem gives an orthonormal eigenbasis. D: **Incorrect:** Rank equals the dimension of the projected subspace, not always one.

**1. Compute the characteristic polynomial:** For \(A=\begin{pmatrix}1&2&1\\1&0&1\\0&0&1\end{pmatrix}\), the block-triangular form gives \[\det(A-\lambda I)=(1-\lambda)((1-\lambda)(-\lambda)-2)=(1-\lambda)(\lambda-2)(\lambda+1).\] Thus eigenvalues include 1 and 2. **2. Test the proposed eigenvectors:** \(A(-3,-1,2)^T=(-3,-1,2)^T\), so this vector has eigenvalue 1. In contrast, \(A(1,1,0)^T=(3,1,0)^T\), which is not a scalar multiple of \((1,1,0)^T\). **Answer: A, B, C — A, B and C.** A: **Correct:** 1 is a root of the characteristic polynomial. B: **Correct:** 2 is also a root. C: **Correct:** Direct multiplication returns the same nonzero vector. D: **Incorrect:** The first two output coordinates require different scalar multipliers.

**1. Impose continuity:** Differentiability at 1 requires matching values: \(p+q+2=r+s\). **2. Match the one-sided slopes:** The left derivative of \(px^2+qx+2\) at 1 is \(2p+q\); the right derivative of \(rx+s\) is r. Therefore \(r=2p+q\), and substitution into continuity gives \(s=2-p\). **3. Compare the formulations:** A and D express these equivalent necessary conditions. B omits the constant 2. C follows only in the special case p=0, not for every differentiable member of the family. **Answer: A, D — A and D.** A: **Correct:** It is the solved form of continuity and derivative matching. B: **Incorrect:** Continuity requires p+q+2, not just p+q. C: **Incorrect:** It unnecessarily assumes p=0; for p=1,q=0, valid values are r=2,s=1. D: **Correct:** It states both the value and slope matching equations.

**1. Identify eigenvectors:** \(A=\begin{pmatrix}0&1&0\\1&0&0\\0&0&1\end{pmatrix}\) swaps the first two coordinates. Vectors \((1,1,0)^T\) and \((0,0,1)^T\) have eigenvalue 1, while \((-1,1,0)^T\) has eigenvalue -1. **2. Form the diagonalization:** These are exactly the columns of B’s matrix P. They are independent, and \(AP=P\operatorname{diag}(1,1,-1)\), proving \(A=PDP^{-1}\). Normalizing the first and third columns produces an orthogonal eigenvector matrix, proving A. **3. Reject the singular candidates:** In C, the first and third columns are negatives of each other, so P is not invertible. The matrix proposed for \(P^{-1}\) in D has the same dependence, so it cannot be an inverse. **Answer: A, B — A and B.** A: **Correct:** Real symmetry, or normalized versions of the displayed eigenvectors, proves orthogonal diagonalizability. B: **Correct:** Its columns are an independent eigenbasis in the stated order. C: **Incorrect:** Its first and third columns are dependent, making it singular. D: **Incorrect:** An inverse cannot be singular, but the proposed matrix is singular.

**1. Compute the least-squares slope:** For \((0,-2),(2,5),(3,4)\), \(n=3,\sum x=5,\sum y=7,\sum x^2=13,\sum xy=22\). Thus \[m=\frac{3(22)-5(7)}{3(13)-5^2}=\frac{31}{14}.\] **2. Compute the intercept:** \(b=\bar y-m\bar x=7/3-(31/14)(5/3)=-19/14\). The fitted line is \(\hat y=(31x-19)/14\). **3. Evaluate the predictions:** At x=11, \(\hat y=(341-19)/14=23\). At x=7, \(\hat y=(217-19)/14=99/7\). **Answer: C, D — C and D.** A: **Incorrect:** It has the wrong slope 13/14. B: **Incorrect:** It has the correct slope but the wrong intercept -91/14. C: **Correct:** It gives the fitted prediction 23 at x=11. D: **Correct:** It gives the fitted prediction 99/7 at x=7.

**1. Distinguish transpose and conjugate transpose:** \(A=\begin{pmatrix}1&i&-i\\i&1&0\\-i&0&1\end{pmatrix}\) satisfies \(A^T=A\). Its conjugate transpose changes the signs of its imaginary entries, so \(A^*\ne A\). **2. Test unitarity and column orthogonality:** Its first column has squared Hermitian norm 3, not 1, so A is not unitary. Also, with \(c_2=(i,1,0)^T\), \(c_3=(-i,0,1)^T\), \(c_2^*c_3=(-i)(-i)=-1\ne0\). Thus not all columns are orthogonal. **Answer: B, D — B and D.** A: **Incorrect:** Conjugation changes the nonzero imaginary off-diagonal entries. B: **Correct:** Ordinary transposition leaves the matrix unchanged. C: **Incorrect:** Its first column does not have unit norm. D: **Correct:** The second and third columns have Hermitian inner product -1.

**1. Express A through a real symmetric matrix:** Write \(A=I+iK\), where \(K=\begin{pmatrix}0&1&-1\\1&0&0\\-1&0&0\end{pmatrix}\). Its characteristic roots are \(0,\sqrt2,-\sqrt2\), since \(\det(\mu I-K)=\mu(\mu^2-2)\). **2. Transfer the eigenpairs:** If \(Kv=\mu v\), then \(Av=(1+i\mu)v\). Hence A has the three distinct eigenvalues \(1,1+i\sqrt2,1-i\sqrt2\). Their product is 3, so A is invertible, with rank 3 and nullity 0. **Answer: A, B — A and B.** A: **Correct:** It corresponds to the zero eigenvalue of K. B: **Correct:** It corresponds to K’s positive eigenvalue √2. C: **Incorrect:** The determinant is 3, so the rank is 3, not 2. D: **Incorrect:** Invertibility gives nullity zero, not one.

**1. Evaluate the straightforward sequences:** \(-1+n/(n+1)=-1/(n+1)\to0\). Also \(n!\ge2^{n-1}\) for n≥1, so the factorial sequence diverges to positive infinity. **2. Show the sine sequence cannot converge:** Suppose \(\sin n\to L\) in radians. The identity \(\sin(n+2)=2\cos1\sin(n+1)-\sin n\) gives \(L=2\cos1\,L-L\), so L=0. Then \(\cos n=[\sin(n+1)-\cos1\sin n]/\sin1\to0\), contradicting \(\sin^2n+\cos^2n=1\). Thus sin n does not converge, and in particular does not tend to zero. **Answer: A, C, D — A, C and D.** A: **Correct:** It simplifies to a reciprocal tending to zero. B: **Incorrect:** The recurrence and trigonometric identity rule out convergence to zero. C: **Correct:** The factorial grows without bound. D: **Correct:** The contradiction proves nonconvergence.

**1. Evaluate value and gradient at the base point:** For \(f(x,y)=\ln(x^2+y^2)\), \[\nabla f=\left(\frac{2x}{x^2+y^2},\frac{2y}{x^2+y^2}\right).\] At (1,2), \(f=\ln5\) and \(\nabla f=(2/5,4/5)\). **2. Use the coordinate increments:** The displacement to (1.02,2.01) is (0.02,0.01). The linear estimate is \[L=\ln5+\frac25(0.02)+\frac45(0.01)=\ln5+0.016=1.625437912\ldots.\] **3. Round the approximation:** To four decimal places, enter 1.6254. The question requests the tangent estimate, not direct evaluation of the original logarithm. **Answer: 1.6254.**

**1. Identify clustering:** Clustering groups similar observations using their measured features without requiring preassigned target labels. **2. Match the hospital’s use:** Patients with similar symptoms and laboratory profiles can be grouped to explore patterns. Predicting recovery time or a known disease label would instead use supervised prediction. **Answer: C — Grouping patients with similar symptoms and test results without prior labels..** A: **Incorrect:** A continuous recovery-time target calls for regression. B: **Incorrect:** Known disease labels make this supervised classification. C: **Correct:** It groups similar patients without prior labels, which is clustering. D: **Incorrect:** Variable-importance analysis is a different objective, not the definition of clustering.