Machine Learning Foundations · Quiz 2 · Jan 2026

**1. Identify the projection:** Put \(P=uu^*\). Since \(u^*u=1\), \(P^*=P\) and \(P^2=u(u^*u)u^*=P\). Thus P is the orthogonal projection onto the span of u. **2. Check the adjoint:** For \(A=I-2P\), \(A^*=I-2P^*=A\). Therefore A is Hermitian. **3. Check norm preservation:** \[A^*A=A^2=(I-2P)^2=I-4P+4P^2=I.\] Hence A is also unitary. This is a Householder reflection: it reverses the u direction and fixes its orthogonal complement. **Answer: A — A is Hermitian and unitary.** A: **Correct:** The projection identities establish both A*=A and A*A=I. B: **Incorrect:** The Hermitian part is true, but A is unitary as well. C: **Incorrect:** The unitary part is true, but A is also Hermitian. D: **Incorrect:** Both stated properties hold.

**1. Multiply the original eigenvalues:** The determinant equals the product of eigenvalues, so \(\det A=2\cdot3\cdot5=30\). **2. Scale every dimension:** Multiplying a 3 by 3 matrix by 2 multiplies each of its three eigenvalues by 2. Therefore \[\det(2A)=2^3\det A=8(30)=240.\] Equivalently, multiply the scaled eigenvalues 4, 6 and 10. **Answer: A — 240.** A: **Correct:** Scaling all three dimensions gives 240. B: **Incorrect:** It multiplies the determinant by 2 rather than 2³. C: **Incorrect:** It does not equal the product of the scaled eigenvalues. D: **Incorrect:** It does not include the original determinant and scaling factor correctly.

**1. Impose conjugate symmetry:** A Hermitian matrix satisfies \(A_{12}=\overline{A_{21}}\). For the displayed matrix, \[a+i=\overline{b-i}=\overline b+i,\] so the general condition is \(a=\overline b\). The diagonal entries 2 and 3 are already real. **2. Match the real-parameter option:** For real a and b, conjugation leaves b unchanged and the condition reduces to a=b. Option C states exactly this valid real-parameter family. **Answer: C — a=b with a,b real.** **Convention note:** If complex parameters are allowed, the full necessary and sufficient condition is a=conjugate(b), which does not force them to be real. C gives the intended real-parameter condition and a valid sufficient family. A: **Incorrect:** a−b=2 contradicts the required equality for real parameters. B: **Incorrect:** a=−b does not generally imply conjugate symmetry; for real parameters it works only in the special case a=b=0. C: **Correct:** Equal real values make the off-diagonal entries a+i and a−i conjugates. D: **Incorrect:** With a=1 and b=0 the off-diagonal entries are not conjugates.

**1. Use the quadratic form:** For \(A=\begin{pmatrix}k&b\\b&1\end{pmatrix}\), \[\begin{pmatrix}x&y\end{pmatrix}A\begin{pmatrix}x\\y\end{pmatrix}=(y+bx)^2+(k-b^2)x^2.\] **2. Require strict positivity:** If k>b², every nonzero vector gives a positive value: either x is nonzero and the second term is positive, or x=0 and y² is positive. If k≤b², taking (x,y)=(1,−b) gives a nonpositive value. **3. Check with Sylvester’s criterion:** The determinant is k−b². The condition k>b² also implies k>0, so both leading principal minors are positive. **Answer: B — k>b².** A: **Incorrect:** k<b² makes the determinant negative, so the form is indefinite. B: **Correct:** k>b² is necessary and sufficient for strict positive definiteness. C: **Incorrect:** k<0 already makes the quadratic form negative at (1,0). D: **Incorrect:** For real b, b²≥0 makes k<0 and k>b² mutually impossible.

**1. Apply the Taylor formula:** For \(f(x)=(1-x)^{-1}\), \(f^{(r)}(0)=r!\). Thus the coefficient \(f^{(r)}(0)/r!\) equals 1 for every degree r. **2. Retain degrees zero through three:** \[f(x)=1+x+x^2+x^3+O(x^4).\] The full geometric series converges for |x|<1; the requested third-degree Taylor polynomial is its first four terms. **Answer: A — 1+x+x²+x³.** A: **Correct:** It has the correct constant, linear, quadratic and cubic coefficients. B: **Incorrect:** Alternating signs correspond to 1/(1+x), not 1/(1−x). C: **Incorrect:** Its quadratic and cubic signs are wrong. D: **Incorrect:** It omits the constant term and includes a fourth-degree term.

**1. Prove the first Gram matrix is positive semidefinite:** For every \(z\in\mathbb R^n\), \[z^TA^TAz=\|Az\|^2\ge0.\] AᵀA is also symmetric, so it is positive semidefinite for any real m by n matrix A. **2. Prove the second claim:** For every \(w\in\mathbb R^m\), \(w^TAA^Tw=\|A^Tw\|^2\ge0\). No square-shape or symmetry assumption on A is needed. **3. Check the equality claim:** AᵀA is n by n and AAᵀ is m by m, so they need not even have the same shape. Even when A is square, they need not be equal unless A is normal. **Answer: A, B — AᵀA and AAᵀ are both positive semidefinite.** A: **Correct:** Its quadratic form is a squared Euclidean norm. B: **Correct:** Its quadratic form is also a squared Euclidean norm. C: **Incorrect:** Symmetry of A is not necessary; the result holds for rectangular matrices too. D: **Incorrect:** A need not be square for AAᵀ to be positive semidefinite. E: **Incorrect:** The products can differ in shape, or differ in entries even for square A.

**1. Compute the adjoint product:** Let \(C=\begin{pmatrix}1&i\\i&1\end{pmatrix}\). Direct multiplication gives \(C^*C=2I\). For real nonzero m, \[B^*B=\frac{2}{m^2}I.\] **2. Solve the unitary condition:** B is unitary exactly when m²=2, that is m=√2 or m=−√2. In particular, the listed m=√2 condition is correct, while m=2 gives B*B=I/2. **3. Verify the eigenvalue magnitudes:** The eigenvalues of C are 1+i and 1−i, so those of B are \((1\pm i)/m\), with magnitude \(\sqrt2/|m|\). At m=√2 both have modulus 1; this is not true for arbitrary nonzero m. **Answer: C, D — B is unitary at m=√2, and then all its eigenvalues have modulus 1.** A: **Incorrect:** Only m=±√2 gives a unitary matrix among real nonzero m. B: **Incorrect:** m=2 gives B*B=I/2, not I. C: **Correct:** m=√2 normalizes both orthogonal columns to unit length. D: **Correct:** A unitary matrix has all eigenvalues on the unit circle; the explicit eigenvalues confirm it. E: **Incorrect:** The eigenvalue magnitudes are √2/|m| and therefore depend on m.

**1. Recall the descent direction:** Gradient descent updates \(x_{t+1}=x_t-\eta f\prime(x_t)\). Just left of the local minimum a, the plotted curve has negative slope, so the update moves right, toward a. **2. Trace the second initialization:** Just right of b, the plotted slope is positive. Subtracting a small positive multiple of that slope moves left, toward b. **3. Use the small-step assumption:** With the stated very small positive displacement and learning rate, each initialization stays in its nearby basin and approaches its own local minimum. The deeper minimum at b does not attract every initialization across the separating local maximum. **Answer: A, B — instance (1) converges to a; instance (2) converges to b.** **Convention note:** The convergence conclusion uses the smooth local minima shown in the graph and the stated sufficiently small step size; a large arbitrary learning rate would not carry the same guarantee. A: **Correct:** Starting just left of a moves into and converges within its local basin. B: **Correct:** Starting just right of b moves left into its local basin. C: **Incorrect:** The second initialization is in the basin near b, so both do not approach a. D: **Incorrect:** The first initialization remains in the basin near a, despite b being deeper. E: **Incorrect:** The negative slope left of a sends the iterate right rather than toward negative infinity. F: **Incorrect:** The positive slope right of b sends the iterate left rather than toward positive infinity.

**1. Connect rank with the SVD:** In an SVD, orthogonal factors do not change rank. The rank is therefore the number of strictly positive singular values on the diagonal of the singular-value matrix. **2. Count the positive entries:** The list has positive singular values 5, 3 and 1; every other listed value is zero. Thus the rank is 3. **Answer: 3.** **Convention note:** A 10 by 7 matrix conventionally has seven singular values counted with multiplicity, while the printed list includes an extra zero. This redundant zero does not affect the rank.

**1. Compute the row Gram matrix:** For \(A=\begin{pmatrix}0&1&0\\1&0&0\end{pmatrix}\), the two rows are orthonormal, giving \(AA^T=I_2\). **2. Extract singular values:** The eigenvalues of AAᵀ are 1 and 1, so the two singular values are their square roots, 1 and 1. **3. Add them:** Their sum, also called the nuclear norm, is 1+1=2. The matrix is rectangular, so its singular-value sum should not be inferred from a square-matrix trace. **Answer: 2.**

**1. Find the exact minimizer:** \(f(x)=x^2-6x+5=(x-3)^2-4\), so its unique minimizer is \(x^*=3\). Equivalently, f′(x)=2x−6 vanishes there. **2. Perform the stated single update:** At x⁰=4, the gradient is 2. With η=0.1, \[x^1=4-0.1(2)=3.8.\] **3. Measure distance from the minimizer:** \(|x^1-x^*|=|3.8-3|=0.8\), already written to one decimal place. **Answer: 0.8.**

**1. Compute the mean:** For (2,2), (−2,−2), (1,−1), (−1,1), both coordinate sums vanish. Hence the mean is (0,0), and the observations are already centered. **2. Form empirical covariance:** The sums of squared coordinates are both 10; the cross-product sum is 4+4−1−1=6. Using divisor n=4, \[C=\frac14\begin{pmatrix}10&6\\6&10\end{pmatrix}=\begin{pmatrix}2.5&1.5\\1.5&2.5\end{pmatrix}.\] **3. Sum all entries:** \(a+b+c+d=2.5+1.5+1.5+2.5=8\). **Answer: 8.** **Convention note:** PCA covariance uses the empirical divisor n here. The unbiased sample-covariance divisor n−1 would produce a different entry sum, 32/3.

**1. Build the centered covariance:** The four points (2,2), (−2,−2), (1,−1), (−1,1) have mean zero and empirical covariance \(C=\begin{pmatrix}2.5&1.5\\1.5&2.5\end{pmatrix}\). **2. Find its principal directions:** The unit vectors \(u=(1,1)^T/\sqrt2\) and \(v=(1,-1)^T/\sqrt2\) have eigenvalues 4 and 1 respectively. Thus v corresponds to the smaller eigenvalue. **3. Evaluate directional variance:** For a unit direction v the variance is \(v^TCv\). Since Cv=v, \(v^TCv=v^Tv=1\). Directly, the projected values are 0, 0, √2 and −√2, whose mean square is 1. **Answer: 1.** **Convention note:** Use the same divisor-n covariance convention as the preceding PCA question. Changing the eigenvector sign does not change its variance.