Machine Learning Foundations · Quiz 1 · Sep 2025

**1. Identify the prediction target:** Loan approval is a discrete choice: approve or do not approve. Income, score and records are input features. **2. Match the task:** Predicting one of these classes is binary classification. Regression predicts a continuous value, while clustering and dimensionality reduction serve different purposes. **Answer: B — Classification.** A: **Incorrect:** It would fit a numerical target such as the loan amount, not the stated yes/no outcome. B: **Correct:** It predicts one of two discrete decision categories. C: **Incorrect:** It groups applicants without directly learning the approval labels. D: **Incorrect:** It compresses features rather than directly defining the requested decision task.

**1. Fit the intercept and symmetric points:** The samples are \((-1,2),(0,1),(1,2)\). In \(y=\theta_0+\theta_1x+\theta_2x^2\), the middle point gives \(\theta_0=1\). The outer points give \(-\theta_1+\theta_2=1\) and \(\theta_1+\theta_2=1\). **2. Solve and justify least squares:** Adding gives \(\theta_2=1\), then \(\theta_1=0\). Thus \(y=x^2+1\) fits all three points exactly, attaining the minimum possible squared loss zero. Distinct x values make the quadratic fit unique. **Answer: A — y=x²+1.** A: **Correct:** It predicts 2,1,2 and has zero loss. B: **Incorrect:** It predicts 1,0,1, missing each label by one. C: **Incorrect:** It predicts 0,-1,0, missing each label by two. D: **Incorrect:** It predicts -2,0,2 and fails the first two samples.

**1. Compute the dot product:** \(v_1=(1,2,2)^T\), \(v_2=(2,-1,0)^T\), and \(v_1^Tv_2=2-2+0=0\). Both vectors are nonzero, so they are orthogonal and cannot be parallel. **2. Apply the projection formula:** \(\operatorname{proj}_{v_2}(v_1)=v_2(v_2^Tv_1)/(v_2^Tv_2)=0\). Reversing the roles also gives zero because the numerator remains the same zero dot product. **Answer: B, C, D — B, C and D.** A: **Incorrect:** Nonzero orthogonal vectors are not parallel. B: **Correct:** Their dot product is zero. C: **Correct:** Its projection coefficient has zero numerator. D: **Correct:** The reverse projection coefficient also has zero numerator.

**1. Encode the samples:** With \(f(x)=0.5x_1+0.5x_2-x_3\), the samples \((2,4,3),(5,5,6),(2,2,1),(4,4,6)\) encode to \(0,-1,1,-2\), respectively. The decoder repeats each scalar three times. **2. Compute squared distances:** The errors are \(2^2+4^2+3^2=29\); \(6^2+6^2+7^2=121\); \(1^2+1^2+0^2=2\); and \(6^2+6^2+8^2=136\). **3. Average across samples:** \(L=(29+121+2+136)/4=288/4=72\). This is the mean squared Euclidean distance per sample. **Answer: 72.**

**1. Construct the required matrix:** If every nonzero vector has eigenvalue 2, then \(Av=2v\) for all vectors, which forces \(A=2I_{10}\). This matrix does satisfy the requirement. **2. Evaluate its determinant:** A diagonal determinant is the product of diagonal entries, so \(\det A=2^{10}=1024\). Since the matrix exists, do not enter the fallback answer -1. **Answer: 1024.**

**1. Express the column sum as a matrix product:** If \(c_1,\ldots,c_n\) are the columns, then \(c_1+\cdots+c_n=A\mathbf1\), where \(\mathbf1=(1,\ldots,1)^T\ne0\). **2. Apply the eigenvalue definition:** The hypothesis gives \(A\mathbf1=0=0\mathbf1\). Thus \(\mathbf1\) is a nonzero eigenvector corresponding to \(\lambda=0\). **Answer: 0.**

**1. Read the four plotted samples:** They are \((2,0),(3,3),(5,5),(6,8)\). For \(f(x)=2x-4\), the predictions are \(0,2,6,8\), yielding squared errors \(0,1,1,0\) and total 2. **2. Compare all offered models:** Constants 4 and 8 give squared-error totals \(16+1+1+16=34\) and \(64+25+9+0=98\). The model \(4-2x\) predicts \(0,-2,-6,-8\), giving total \(0+25+121+256=402\). **3. Select the minimum:** The totals are 2, 34, 98 and 402, so A wins. Dividing every total by four leaves the ranking unchanged. **Answer: A — f(x)=2x-4.** A: **Correct:** Its total squared error is 2. B: **Incorrect:** Its total squared error is 34. C: **Incorrect:** Its total squared error is 98. D: **Incorrect:** Its total squared error is 402.

**1. State what zero loss requires:** A sum of squared residuals is zero only if every residual is zero. Hence a line \(wx+b\) must pass through all four plotted points. **2. Test collinearity:** The slope from \((2,0)\) to \((3,3)\) is 3, whereas the slope from \((3,3)\) to \((5,5)\) is 1. A single line cannot have both slopes, so no linear model fits all samples exactly. **Answer: A — No.** A: **Correct:** The samples are not collinear, so zero linear training loss is impossible. B: **Incorrect:** It contradicts the unequal-slope test. C: **Incorrect:** The scaled plot supplies sufficient coordinates to decide.

**1. Establish row independence:** For the displayed matrix, \(\det A=-4\ne0\), so its three rows are linearly independent. Their only zero linear combination has all coefficients zero. **2. Test the displayed relations:** \(R_2-R_1-R_3=(0,2,2)\), \(R_1+R_2-2R_3=(0,3,5)\), and \(R_2-2R_1-R_3=(-1,2,0)\). None is zero. The trivial relation in C always holds. **Answer: C — 0R₁+0R₂+0R₃=0.** A: **Incorrect:** Its left side is (0,2,2), not zero. B: **Incorrect:** Its left side is (0,3,5), not zero. C: **Correct:** The all-zero coefficient combination is valid. D: **Incorrect:** Subtracting the right side gives (-1,2,0), not zero.

**1. Check orthogonality:** The rotation matrix has columns \((\cos\theta,\sin\theta)^T\) and \((-\sin\theta,\cos\theta)^T\). Each has norm one and their dot product is zero. Therefore \(A^TA=I\). **2. Check invertibility and symmetry:** Orthogonality gives \(A^{-1}=A^T\) and \(\det A=1\). Symmetry would require \(-\sin\theta=\sin\theta\), impossible for \(0<\theta<180^\circ\), where the sine is positive. **Answer: A, B — A and B.** A: **Correct:** Its columns are orthonormal. B: **Correct:** An orthogonal matrix is invertible, with inverse its transpose. C: **Incorrect:** The off-diagonal entries are opposite nonzero numbers.

**1. Compute the characteristic equation:** \(\det(A-\lambda I)=\lambda^2-2\cos\theta\,\lambda+1\). The discriminant is \(-4\sin^2\theta\). **2. Use the strict angle interval:** For \(0<\theta<180^\circ\), \(\sin\theta\ne0\), so the discriminant is negative. The eigenvalues \(\cos\theta\pm i\sin\theta\) are nonreal. **Answer: A — A does not have real eigenvalues.** A: **Correct:** Both eigenvalues are nonreal under the stated strict interval. B: **Incorrect:** The negative discriminant rules out distinct real roots. C: **Incorrect:** It rules out real roots altogether, including a repeated real pair.

**1. Separate diagonalization from invertibility:** Diagonalizability means having a basis of eigenvectors. Invertibility additionally requires every eigenvalue to be nonzero. **2. Give a counterexample:** \(A=\operatorname{diag}(1,0)\) is already diagonal, hence diagonalizable, but its determinant is zero. Therefore the implication is false. **Answer: B — False.** A: **Incorrect:** A diagonalizable matrix can have eigenvalue zero. B: **Correct:** The explicit diagonal singular matrix disproves the universal claim.

**1. Use real symmetry:** A real symmetric matrix also satisfies \(A^*=A\). For a possibly complex eigenvector \(v\ne0\), \(v^*Av=\lambda v^*v\). **2. Show the eigenvalue is real:** The scalar \(v^*Av\) is real because its complex conjugate equals itself. Also \(v^*v>0\), so \(\lambda=(v^*Av)/(v^*v)\) is real. **Answer: A — True.** A: **Correct:** The Hermitian Rayleigh quotient proves every eigenvalue is real. B: **Incorrect:** It contradicts the real-symmetric spectral property.

**1. Identify the valid direction:** Having n distinct eigenvalues is sufficient to produce n independent eigenvectors and hence a diagonalization over a field containing those eigenvalues. **2. Disprove necessity:** \(I_n\), for \(n>1\), is diagonalizable but has only the one distinct eigenvalue 1. The correct criterion is n independent eigenvectors, not n distinct eigenvalues. **Answer: B — False.** A: **Incorrect:** The “only if” direction fails for the identity matrix. B: **Correct:** Repeated eigenvalues can still admit a complete eigenbasis.

**1. Evaluate value and gradient:** For \(f(x_1,x_2)=e^{x_1}+x_2\), \(f(0,0)=1\), and \(\nabla f=(e^{x_1},1)\), giving \(\nabla f(0,0)=(1,1)\). **2. Form the tangent plane:** \(L_{(0,0)}[f](x)=f(0,0)+\nabla f(0,0)\cdot(x_1,x_2)=1+x_1+x_2\). **Answer: A — 1+x₁+x₂.** A: **Correct:** It matches the function value and both first derivatives. B: **Incorrect:** It omits the value 1 and doubles the second slope. C: **Incorrect:** Both slopes have the wrong signs. D: **Incorrect:** It omits the value 1 and negates the second slope.

**1. Write the tangent plane explicitly:** Since \(f=e^{x_1}+x_2\), the approximation is \(z=1+x_1+x_2\). Rearrange it as \(x_1+x_2-z+1=0\). **2. Match the normal representation:** Taking \(w=(1,1,-1)^T\) gives \(w^Tw=3\), with \(b=1\). This is the orientation used by the supplied answer. **3. Account for orientation:** Multiplying the entire plane equation by -1 gives the equally valid \(w=(-1,-1,1)^T\), \(b=-1\), still with \(w^Tw=3\). The normalization fixes the magnitude, not the sign. **Answer: 1.** **Convention note:** The question does not orient the normal, so b=1 and b=-1 both represent the plane with the stated norm. The returned value 1 follows the source convention; it is not uniquely determined without a sign convention.

**1. Solve for a level curve:** Setting \(e^{x_1}+x_2=c\) gives \(x_2=c-e^{x_1}\). Its slope and second derivative are both \(-e^{x_1}<0\), so it decreases and bends downward. **2. Check intercepts and placement:** At \(x_1=0\), levels \(c=2,1,-1\) cross \(x_2=1,0,-2\), respectively. As \(x_1\to-\infty\), the curves approach heights c. Panel A has these properties; panel B rises instead. **Answer: A — (A).** A: **Correct:** It shows decreasing curves with the correct level intercepts. B: **Incorrect:** Its increasing curves have the wrong derivative sign.