Machine Learning Techniques · Quiz 1 · Jan 2025

**1. Assignment step:** Each point selects its nearest current mean, so quantity (1) is no greater than (2). **2. Mean update:** The arithmetic mean minimizes within-cluster squared distance for fixed assignments, so (3) is no greater than (1). Hence \((3)\le(1)\le(2)\). **Answer: C — \((3)\le(1)\le(2)\), with C the intended strict-progress choice.** **Convention note:** The choices use strict inequalities, but equality is possible, particularly at convergence. C is the intended ordering; none is universally true with strict inequality. A: **Incorrect:** The assignment step cannot increase the objective. B: **Incorrect:** The mean update cannot increase the objective. C: **Correct:** This is the intended decreasing order when both steps make strict progress. D: **Incorrect:** It reverses the old and updated objective ordering.

**1. Use the shape of a cluster cell:** With fixed means, a nearest-centroid Voronoi cell is an intersection of half-spaces and is convex. **2. Take the segment:** \((0.2,1)=0.8(0,1)+0.2(1,1)\). It lies between the two points known to belong to the same cell; the other proposed points are outside that guaranteed segment. **Answer: C — (0.2,1).** **Convention note:** This assumes nearest-centroid assignment with a consistent tie rule; the statement concerns a candidate data point at that location. A: **Incorrect:** It extrapolates beyond the known segment and need not be in the cell. B: **Incorrect:** It lies off the segment and need not be in the cell. C: **Correct:** It is a convex combination of the two given points. D: **Incorrect:** It lies off the segment and need not be in the cell.

**1. Obtain the feature-space direction:** Let \(X^TX\alpha=\lambda\alpha\), \(\|\alpha\|=1\), and \(\lambda>0\). Then \(XX^T(X\alpha)=\lambda X\alpha\). **2. Normalize:** \(\|X\alpha\|^2=\alpha^TX^TX\alpha=\lambda\), so the unit PC is \(w=X\alpha/\sqrt\lambda\). **3. Project:** The scalar coordinate of \(x_i\) is \(x_i^Tw=x_i^TX\alpha/\sqrt\lambda\). **Answer: C — \(x_i^TX\alpha/\sqrt\lambda\).** A: **Incorrect:** The d-dimensional point cannot directly multiply the n-dimensional Gram eigenvector when d>n. B: **Incorrect:** It has the same dimension mismatch and an unjustified extra factor. C: **Correct:** It uses X to lift the eigenvector and divides by its norm. D: **Incorrect:** It omits the normalization by the square root of the eigenvalue.

**1. Apply the Gram criteria:** A real matrix is a Gram matrix exactly when it is symmetric positive semidefinite. Positive definite matrices qualify. **2. Check A and D:** Their leading principal minors are respectively \((4,19,70)\) and \((3,11,52)\), all positive. Sylvester’s criterion establishes positive definiteness. **3. Exclude B and C:** B has negative diagonal entries, impossible for squared vector norms. C is not symmetric. **Answer: A, D — A and D.** A: **Correct:** It is symmetric positive definite, with leading minors 4,19,70. B: **Incorrect:** A Gram diagonal entry is a squared norm and cannot be negative. C: **Incorrect:** Its (1,2) and (2,1) entries differ. D: **Correct:** It is symmetric positive definite, with leading minors 3,11,52.

**1. Check kernels:** \((x^Ty)^3\) is an inner product of tensor-cube features. \(e^{-\|x-y\|^2}\) is the Gaussian positive-semidefinite kernel. **2. Check the matrix claims:** Both traces equal the sum of squared entries, \(\operatorname{tr}(XX^T)=\operatorname{tr}(X^TX)=\sum_{ij}X_{ij}^2\). Their eigenvectors generally inhabit different spaces; shared nonzero eigenvalues do not imply identical eigenvectors. **Answer: A, B, C — A, B and C.** A: **Correct:** The tensor feature map proves kernel validity. B: **Correct:** This is the Gaussian kernel. C: **Correct:** Both traces equal the squared Frobenius norm of X. D: **Incorrect:** The eigenvectors need not coincide and can even have different dimensions.

**1. Check normalization:** \(w=(1,1)/\sqrt2\) has unit length. **2. Take the scalar projection:** \(w^T(3,4)=7/\sqrt2\approx4.949747\), which rounds to 4.95. **Answer: 4.95.**

**1. Count original storage:** \(1000\times20=20000\) numbers. **2. Count PCA storage:** Five coordinates per point require 5000 numbers; five 20-dimensional basis vectors require 100. Total: 5100. **3. Calculate the reduction:** \((20000-5100)/20000\times100=74.50\%\). **Answer: 74.50.** **Convention note:** This includes the basis and excludes a separately stored mean. Counting only reduced coordinates gives 75%; storing an additional 20-number mean gives 74.40%.

**1. Exclude the already selected seed:** From (1,1), the other points (2,5),(2,4),(0,3),(1,0) have Euclidean distances \(\sqrt{17},\sqrt{10},\sqrt5,1\). **2. Identify the smallest positive weight:** Point (1,0) has distance 1. **3. Normalize the stated distance weights:** \[p=\frac1{\sqrt{17}+\sqrt{10}+\sqrt5+1}\approx0.09504,\] giving 0.10 to two decimals. **Answer: 0.10.** **Convention note:** The question explicitly requests Euclidean-distance scores. Standard K-means++ uses squared distances, which would give 1/33≈0.03030 instead. The selected seed itself has probability zero; “least chance” is interpreted among unselected candidates.

**1. Combine the observations:** Your four tails and your friend’s sequence TTTTH give eight tails and one head. **2. Form the likelihood:** \(L(p)=(1-p)^4(1-p)^4p=p(1-p)^8\). There is no binomial coefficient because the stopping sequence is specified; any constant would not change the maximizer. **3. Maximize:** \(\ell\prime(p)=1/p-8/(1-p)=0\) gives \(\hat p=1/9\approx0.11111\), or 0.11. **Answer: 0.11.**

**1. Apply Bayes’ rule:** The first-component responsibility is \(r_{11}=\pi_1\mathcal N(1;1,1)/[\pi_1\mathcal N(1;1,1)+\pi_2\mathcal N(1;3,1)]\). **2. Cancel common factors:** Both mixing weights and normal-density constants match. The squared deviations are 0 and 4, giving \(r_{11}=1/(1+e^{-2})\approx0.880797\). Rounded: 0.88. **Answer: 0.88.**

**1. Find the data direction:** All points are multiples of (1,2), whose length is \(\sqrt5\). **2. Normalize:** The unit representative along the line is \((1,2)/\sqrt5\). The lengths of A, B, C and D are respectively \(\sqrt5/3,1,3,1\); D is unit length but points along (3,4), not along the data line. **Answer: B — \((1,2)^T/\sqrt5\).** **Convention note:** D also has unit length. B is uniquely correct only when “representative” means a direction of the given collinear dataset; if the question asks merely for any unit vector, both B and D qualify. A: **Incorrect:** It has length √5/3, not 1. B: **Correct:** It has length 1 and follows the collinear data. C: **Incorrect:** It is three times the unit direction and has length 3. D: **Incorrect:** It has unit length but is not a representative direction of this dataset.

**1. Write the likelihood:** Ten defective and ninety nondefective items give \(L(p)\propto p^{10}(1-p)^{90}\). **2. Multiply by the prior:** Beta(1,1) is uniform, so the posterior density has the same powers. Comparing with \(p^{\alpha-1}(1-p)^{\beta-1}\) gives Beta(11,91). **Answer: B — Posterior distribution for p is Beta(11, 91)..** A: **Incorrect:** It interchanges defective and nondefective counts. B: **Correct:** Adding the counts to the prior gives Beta(11,91). C: **Incorrect:** The unknown probability is continuous; its posterior is not a Bernoulli outcome distribution. D: **Incorrect:** This is also a discrete outcome distribution rather than the posterior over p.