Machine Learning Techniques · Quiz 1 · May 2026

Because \(x_1=x_2=x_3\), their covariance block is \[ 100\begin{bmatrix}1&1&1\\1&1&1\\1&1&1\end{bmatrix}. \] It has eigenvalues \(300,0,0\), with leading unit eigenvector \((1,1,1)^T/\sqrt3\). Features \(x_4,\ldots,x_8\) contribute five eigenvalues \(10\); the two constant-zero features contribute two more zeros. Thus the spectrum is \[ 300,\underbrace{10,\ldots,10}_{5},\underbrace{0,\ldots,0}_{4}. \] Exactly four eigenvalues vanish, and the first principal direction is \(\pm(1,1,1,0,\ldots,0)^T/\sqrt3\). A: There are four zero eigenvalues, not three, and e₁ alone is not the principal direction. B: Correct: four zero eigenvalues and the normalized shared-feature direction, up to an arbitrary overall sign. C: The zero count is correct but the principal direction is not e₁. D: Misses the two zero modes arising from the identical first three features.

Let \(F(v)=\sum_i\|3x_i-2v\|^2\). Differentiating gives \[ \nabla F(v)=-4\sum_i(3x_i-2v) =-12\sum_i x_i+8nv. \] Set the gradient to zero: \[ \boxed{v^*=\frac{3}{2n}\sum_i x_i}. \] The Hessian is \(8nI\), which is positive definite for \(n>0\), so this minimizer is unique. A: Omits the factor 3/2 arising from 3xᵢ−2v. B: Correct: the minimizer is 3/2 times the sample mean. C: Twice the correct minimizer. D: Uses the reciprocal scaling rather than 3/2.

Use the **given** centroids \(0\) and \(10\); the expression is evaluated after assignment and before updating the centroids.

Therefore the objective is \(0+4+16+4=\boxed{24}\). A: Would be the objective after updating centroids to 1 and 7, which the expression does not request. B: Does not equal the sum using the given centroids. C: Correct: the fixed-centroid assignment cost is 24. D: Does not equal the nearest-centroid squared-distance sum.

The objective is a sum of nonnegative squared distances. With \(k=1000\) and \(1000\) data points, one can place one centroid at every point, giving \[ J=\sum_{i=1}^{1000}\|x_i-x_i\|^2=\boxed0. \] No smaller value is possible. Among the listed choices, **\(k=1000\)** therefore permits the minimum objective. A: A single centroid must usually represent many distinct points, producing positive error. B: More flexible than k=1 but does not in general fit every point exactly. C: More flexible than k=10, but still need not give zero error. D: Correct: one centroid per point gives the global lower bound zero.

For a scalar feature map of the form \(\phi(x)=cx^2\), \[ \phi(x_1)^T\phi(x_2)=c^2x_1^2x_2^2. \] Matching \(10x_1^2x_2^2\) requires \(c^2=10\). The listed valid map is \[ \boxed{\phi(x)=\sqrt{10}\,x^2}. \] A: Produces coefficient 100 rather than 10. B: Produces coefficient 1 rather than 10. C: Produces 10x₁x₂ instead of 10x₁²x₂². D: Correct: multiplying the two features gives exactly the specified kernel.

Jensen's inequality applies to a convex combination: \[ \lambda_i\ge0,\qquad \lambda_1+\lambda_2+\lambda_3=1. \] Both \((1/2,1/4,1/4)\) and \((1/3,1/3,1/3)\) meet these conditions. The other choices fail for some convex functions. With \(f(t)=t^2\), choose \((a_1,a_2,a_3)=(1,0,0)\) for A, \((1,1,1)\) for C, and \((0,1,0)\) for D; in each case the proposed inequality is false. A: Has a negative weight; f(t)=t² and (1,0,0) would require 1≤−1. B: Correct: all weights are nonnegative and their sum is 1. C: Weights sum to 3; f(t)=t² and all aᵢ=1 would require 9≤3. D: Has negative weights; f(t)=t² and (0,1,0) would require 25≤−5. E: Correct: equal nonnegative weights sum to 1.

**Use closure properties of positive semidefinite kernels.** The RBF kernel satisfies \(K_2(x,x)=1\), so A is \(K_1K_2\), a valid product kernel. For B, expand the exponent: \[ e^{-(\|x\|^2+\|y\|^2)/(2\sigma^2)}e^{x^Ty/\sigma^2} =e^{-\|x-y\|^2/(2\sigma^2)}=K_2(x,y). \] Thus B is the valid sum \(K_1+K_2\). C is the pointwise product of the valid kernel \(K_1+K_2\) with itself. D fails already at \(x=y=0\): \(K_1(0,0)-K_2(0,0)=-1\), whereas a PSD kernel must have a nonnegative diagonal. A: Correct: normalization leaves the RBF unchanged, and its product with the linear kernel is PSD. B: Correct: the exponential product is exactly the RBF kernel, so this is a sum of two PSD kernels. C: Correct: the pointwise square is a kernel product; the Schur product theorem preserves PSD. D: Incorrect: its value at (0,0) is negative, violating PSD.

The responsibility equation gives \[ \gamma_1=\frac{\pi_1f(x\mid z=1)}{p(x)}=0.8,\qquad\pi_1=0.2. \] Therefore the following quantities are fixed:

The absolute scale \(p(x)\) cannot be recovered from these two ratios alone. A: Not determined: the supplied information fixes density ratios, not the absolute marginal density. B: Correct: γ₁/π₁=4. C: Correct: γ₁/(1−γ₁)=4. D: Correct: all responsibilities sum to one, leaving 0.2.

Write \(\pi_1=p\), so \(\pi_2=1-p\). Since \(f(x\mid1)=3f(x\mid2)\), \[ 0.60=\frac{3p}{3p+(1-p)}=\frac{3p}{1+2p}. \] Hence \(0.6+1.2p=3p\), so \[ p=\frac{0.6}{1.8}=\frac13\approx\boxed{0.33}. \]

The dataset has mean zero. With the covariance convention \[ S=\frac1n\sum_{i=1}^n x^{(i)}x^{(i)T}, \] the requested expression equals \(u^TSu\). A unit direction of maximum variance is an eigenvector for the largest eigenvalue, so \[ u^TSu=\lambda_1=100^{\,2-1}=\boxed{100}. \] The normalization is the \(1/n\) covariance convention used by the question's PCA expression.