Machine Learning Practice · Quiz 2 · Sep 2024

**Answer: E** **Topic:** Regularized Regression — *Ridge Regression Coefficients on Collinear Features* ### Step-by-Step Solution **Step 1:** Examine the dataset: $X = \begin{bmatrix} 1 & 2 \\ 2 & 4 \\ 3 & 6 \\ 4 & 8 \end{bmatrix}$ has $x_2 = 2x_1$, and $y = [1, 2, 3, 4]^T = x_1$. By default, `Ridge(alpha=10)` centers $X$ and $y$ (`fit_intercept=True`). **Step 2:** Center $x_1$, $x_2$, and $y$ by subtracting their means ($\bar{x}_1 = 2.5, \bar{x}_2 = 5.0, \bar{y} = 2.5$): - $x_{1c} = [-1.5, -0.5, 0.5, 1.5]^T$ with $\|x_{1c}\|^2 = 2.25 + 0.25 + 0.25 + 2.25 = 5$ - $x_{2c} = 2 x_{1c}$ with $\|x_{2c}\|^2 = 20$ - $y_c = x_{1c}$ **Step 3:** Since $x_{2c} = 2x_{1c}$, minimizing $\|w_1 x_{1c} + w_2 (2x_{1c}) - x_{1c}\|^2 + \alpha(w_1^2 + w_2^2) = 5(w_1 + 2w_2 - 1)^2 + 10(w_1^2 + w_2^2)$ gives $w_2 = 2w_1$ and: $$5(5w_1 - 1)(1) + 10w_1 = 0 \implies 35w_1 = 5 \implies w_1 = \frac{1}{7} \approx 0.1429 \approx 0.14, \quad w_2 = \frac{2}{7} \approx 0.2857 \approx 0.29$$ **Step 4:** Thus, `np.round(coefficients, 2)` outputs **`[0.14, 0.29]`**, which matches **E**. **Key Takeaway / Tip:** Because $L_2$ regularization penalizes $w_1^2 + w_2^2$ and $x_2 = 2x_1$ has twice the magnitude of $x_1$, Ridge assigns $w_2 = 2w_1 = 2/7 \approx 0.29$ and $w_1 = 1/7 \approx 0.14$. A: **Incorrect:** Option A (

) does not satisfy the conditions derived in the step-by-step analysis for Ridge Regression Coefficients on Collinear Features (correct choice: E). B: **Incorrect:** Option B (

) does not satisfy the conditions derived in the step-by-step analysis for Ridge Regression Coefficients on Collinear Features (correct choice: E). C: **Incorrect:** Option C (

) does not satisfy the conditions derived in the step-by-step analysis for Ridge Regression Coefficients on Collinear Features (correct choice: E). D: **Incorrect:** Option D (

) does not satisfy the conditions derived in the step-by-step analysis for Ridge Regression Coefficients on Collinear Features (correct choice: E). E: **Correct:** Option E (

) matches our derived result under Ridge Regression Coefficients on Collinear Features. Because $L_2$ regularization penalizes $w_1^2 + w_2^2$ and $x_2 = 2x_1$ has twice the magnitude of $x_1$, Ridge assigns $w_2 = 2w_1 = 2/7 \approx 0.29$ and $w_1 = 1/7 \approx 0.14$.

**Answer: A** **Topic:** Regularized Classification — *ElasticNet l1_ratio Parameter in SGDClassifier* ### Step-by-Step Solution **Step 1:** In `SGDClassifier(penalty='elasticnet', l1_ratio=0.5, alpha=0.01)`, the ElasticNet regularization penalty is a convex combination of $L_1$ (Lasso) and $L_2$ (Ridge) penalties: $$\text{Penalty} = \alpha \left[ \text{l1\_ratio} \cdot \|w\|_1 + (1 - \text{l1\_ratio}) \cdot \frac{1}{2}\|w\|_2^2 \right]$$ **Step 2:** When `l1_ratio=0.5`, both weights `l1_ratio` and `1 - l1_ratio` equal $0.5$. **Step 3:** Therefore, **it assigns equal weightage to L1 and L2 regularizations** (**A**). **Key Takeaway / Tip:** `l1_ratio=1.0` corresponds to pure L1 (Lasso) regularization, `l1_ratio=0.0` corresponds to pure L2 (Ridge) regularization, and `l1_ratio=0.5` weights both equally. A: **Correct:** Option A (It assigns equal weightage to L1 and L2 regularizations.) matches our derived result under ElasticNet l1_ratio Parameter in SGDClassifier. `l1_ratio=1.0` corresponds to pure L1 (Lasso) regularization, `l1_ratio=0.0` corresponds to pure L2 (Ridge) regularization, and `l1_ratio=0.5` weights both equally. B: **Incorrect:** Option B (It applies only L2 regularization.) does not satisfy the conditions derived in the step-by-step analysis for ElasticNet l1_ratio Parameter in SGDClassifier (correct choice: A). C: **Incorrect:** Option C (It applies only L1 regularization.) does not satisfy the conditions derived in the step-by-step analysis for ElasticNet l1_ratio Parameter in SGDClassifier (correct choice: A). D: **Incorrect:** Option D (It disables regularization completely.) does not satisfy the conditions derived in the step-by-step analysis for ElasticNet l1_ratio Parameter in SGDClassifier (correct choice: A).

**Answer: A** **Topic:** Polynomial Regression — *Fitting an Exact Quadratic Function with PolynomialFeatures* ### Step-by-Step Solution **Step 1:** In the given code, $X = [1, 2, 3, 4, 5]^T$ and $y = [1, 4, 9, 16, 25]^T = X^2$. **Step 2:** `PolynomialFeatures(degree=2, interaction_only=False)` transforms each $x_i$ into $[1, x_i, x_i^2]$, which includes the exact quadratic term $x_i^2$. **Step 3:** When `LinearRegression()` is fitted on `(X_poly, y)`, it finds the exact relationship $\hat{y} = 0 + 0 \cdot x + 1 \cdot x^2 = x^2$ with zero residual error. **Step 4:** Therefore, `model.predict(X_poly)` reproduces `y` exactly: **`[1, 4, 9, 16, 25]`** (**A**). **Key Takeaway / Tip:** Because $y = x^2$ lies in the exact span of degree-2 polynomial features $[1, x, x^2]$, Ordinary Least Squares achieves zero training error and returns `[1, 4, 9, 16, 25]`. A: **Correct:** Option A (

) matches our derived result under Fitting an Exact Quadratic Function with PolynomialFeatures. Because $y = x^2$ lies in the exact span of degree-2 polynomial features $[1, x, x^2]$, Ordinary Least Squares achieves zero training error and returns `[1, 4, 9, 16, 25]`. B: **Incorrect:** Option B (

) does not satisfy the conditions derived in the step-by-step analysis for Fitting an Exact Quadratic Function with PolynomialFeatures (correct choice: A). C: **Incorrect:** Option C (

) does not satisfy the conditions derived in the step-by-step analysis for Fitting an Exact Quadratic Function with PolynomialFeatures (correct choice: A). D: **Incorrect:** Option D (

) does not satisfy the conditions derived in the step-by-step analysis for Fitting an Exact Quadratic Function with PolynomialFeatures (correct choice: A).

**Answer: A** **Topic:** Data Preprocessing — *Effect of with_mean=False in StandardScaler* ### Step-by-Step Solution **Step 1:** By default, `StandardScaler(with_mean=True, with_std=True)` centers each feature to zero mean and scales it to unit variance: $z = \frac{x - \mu}{\sigma}$. **Step 2:** When `scaler__with_mean=False` is set, `StandardScaler` **does not subtract the mean** $\mu$, and instead only divides each feature by its standard deviation ($z = \frac{x}{\sigma}$). **Step 3:** Thus, **the data will be standardized without centering, preserving the mean (sign/offset) of the dataset** (**A**). **Key Takeaway / Tip:** `with_mean=False` is especially useful when scaling sparse matrices so that zeros remain zeros without destroying sparsity. A: **Correct:** Option A (The data will be standardized without centering, preserving the mean of the dataset.) matches our derived result under Effect of with_mean=False in StandardScaler. `with_mean=False` is especially useful when scaling sparse matrices so that zeros remain zeros without destroying sparsity. B: **Incorrect:** Option B (The data will be centered but not scaled, leading to variance issues.) does not satisfy the conditions derived in the step-by-step analysis for Effect of with_mean=False in StandardScaler (correct choice: A). C: **Incorrect:** Option C (The parameter has no effect; the data will always be centered regardless of this setting.) does not satisfy the conditions derived in the step-by-step analysis for Effect of with_mean=False in StandardScaler (correct choice: A). D: **Incorrect:** Option D (The scaling operation will be disabled entirely, leaving the data unchanged.) does not satisfy the conditions derived in the step-by-step analysis for Effect of with_mean=False in StandardScaler (correct choice: A).

**Answer: B** **Topic:** Scikit-Learn Utilities — *Identifying Target Types with type_of_target* ### Step-by-Step Solution **Step 1:** Evaluate `type_of_target` on each of the three inputs: 1. `np.array([1, 0, 0, 1])`: a 1D integer array containing only two distinct values (`0` and `1`) $\implies$ **`'binary'`. 2. `np.array([[1, 2], [3, 4]])`: a 2D integer array with values $> 1$ (`1, 2, 3, 4`, not a 0/1 indicator matrix) $\implies$ **`'multiclass-multioutput'`. 3. `[1.5, 2.7, 3.1]`: a 1D list of floating-point numbers $\implies$ **`'continuous'`. **Step 2:** Thus, the output sequence is **`'binary', 'multiclass-multioutput', 'continuous'`**, which matches **B**. **Key Takeaway / Tip:** A 2D array is `'multilabel-indicator'` only if its entries are exclusively binary (`0` and `1`); when entries include integers like `2, 3, 4`, `type_of_target` returns `'multiclass-multioutput'`. A: **Incorrect:** Option A (

) does not satisfy the conditions derived in the step-by-step analysis for Identifying Target Types with type_of_target (correct choice: B). B: **Correct:** Option B (

) matches our derived result under Identifying Target Types with type_of_target. A 2D array is `'multilabel-indicator'` only if its entries are exclusively binary (`0` and `1`); when entries include integers like `2, 3, 4`, `type_of_target` returns `'multiclass-multioutput'`. C: **Incorrect:** Option C (

) does not satisfy the conditions derived in the step-by-step analysis for Identifying Target Types with type_of_target (correct choice: B). D: **Incorrect:** Option D (

) does not satisfy the conditions derived in the step-by-step analysis for Identifying Target Types with type_of_target (correct choice: B).

**Answer: B** **Topic:** Linear Classifiers — *Effect of warm_start=True in Perceptron* ### Step-by-Step Solution **Step 1:** In `scikit-learn` estimators such as `Perceptron`, setting `warm_start=True` tells the model to reuse the learned weights (`coef_` and `intercept_`) from the previous call to `fit()` as initialization when `fit()` is called again. **Step 2:** Compare with the options: - **A is false**: That is the behavior when `warm_start=False` (the default). - **B is true**: 'The model will retain the parameters from the first training session and continue training in the second session.' - **C and D are false**: `warm_start` is a valid, supported parameter in `Perceptron`. **Step 3:** Therefore, **B** is correct. **Key Takeaway / Tip:** `warm_start=True` preserves learned weights across successive `.fit()` calls instead of re-initializing them from scratch. A: **Incorrect:** Option A (

) does not satisfy the conditions derived in the step-by-step analysis for Effect of warm_start=True in Perceptron (correct choice: B). B: **Correct:** Option B (

) matches our derived result under Effect of warm_start=True in Perceptron. `warm_start=True` preserves learned weights across successive `.fit()` calls instead of re-initializing them from scratch. C: **Incorrect:** Option C (

) does not satisfy the conditions derived in the step-by-step analysis for Effect of warm_start=True in Perceptron (correct choice: B). D: **Incorrect:** Option D (

) does not satisfy the conditions derived in the step-by-step analysis for Effect of warm_start=True in Perceptron (correct choice: B).

**Answer: D** **Topic:** Classification Metrics — *Calculating precision_score on Binary Predictions* ### Step-by-Step Solution **Step 1:** Compare `y_true = [0, 1, 1, 0, 1]` and `y_pred = [1, 1, 0, 0, 1]` element by element: - Index 0: `true=0, pred=1` $\implies$ **False Positive (FP)** - Index 1: `true=1, pred=1` $\implies$ **True Positive (TP)** - Index 2: `true=1, pred=0` $\implies$ False Negative (FN) - Index 3: `true=0, pred=0` $\implies$ True Negative (TN) - Index 4: `true=1, pred=1` $\implies$ **True Positive (TP)** **Step 2:** Count True Positives and False Positives: $\text{TP} = 2$ and $\text{FP} = 1$. **Step 3:** Calculate the precision score: $$\text{Precision} = \frac{\text{TP}}{\text{TP} + \text{FP}} = \frac{2}{2 + 1} = \frac{2}{3} \approx 0.67$$ which matches **D**. **Key Takeaway / Tip:** Look only at the indices where `y_pred == 1` (indices 0, 1, 4): 2 out of those 3 have `y_true == 1`, so precision is $2/3 \approx 0.67$. A: **Incorrect:** Option A (0.75) does not satisfy the conditions derived in the step-by-step analysis for Calculating precision_score on Binary Predictions (correct choice: D). B: **Incorrect:** Option B (0.80) does not satisfy the conditions derived in the step-by-step analysis for Calculating precision_score on Binary Predictions (correct choice: D). C: **Incorrect:** Option C (0.50) does not satisfy the conditions derived in the step-by-step analysis for Calculating precision_score on Binary Predictions (correct choice: D). D: **Correct:** Option D (0.67) matches our derived result under Calculating precision_score on Binary Predictions. Look only at the indices where `y_pred == 1` (indices 0, 1, 4): 2 out of those 3 have `y_true == 1`, so precision is $2/3 \approx 0.67$.

**Answer: C** **Topic:** Linear Classifiers — *SGDClassifier with Squared Error Loss and L2 Penalty* ### Step-by-Step Solution **Step 1:** Recall how `SGDClassifier` maps different `loss` functions to standard linear classifiers when combined with `penalty='l2'`: - `loss='hinge'` $\implies$ Linear Support Vector Machine (`SVC`) - `loss='log_loss'` $\implies$ `LogisticRegression` - `loss='perceptron'` $\implies$ `Perceptron` - `loss='squared_error'` with `penalty='l2'` $\implies$ minimizes $\|Xw - y\|_2^2 + \alpha \|w\|_2^2$, which is **`RidgeClassifier`**. **Step 2:** Therefore, **C** (`RidgeClassifier`) is correct. **Key Takeaway / Tip:** Minimizing squared error loss with an $L_2$ penalty via SGD trains a linear model with the exact same objective as `RidgeClassifier`. A: **Incorrect:** Option A (LogisticRegression) does not satisfy the conditions derived in the step-by-step analysis for SGDClassifier with Squared Error Loss and L2 Penalty (correct choice: C). B: **Incorrect:** Option B (Perceptron) does not satisfy the conditions derived in the step-by-step analysis for SGDClassifier with Squared Error Loss and L2 Penalty (correct choice: C). C: **Correct:** Option C (RidgeClassifier) matches our derived result under SGDClassifier with Squared Error Loss and L2 Penalty. Minimizing squared error loss with an $L_2$ penalty via SGD trains a linear model with the exact same objective as `RidgeClassifier`. D: **Incorrect:** Option D (Support Vector Machine (SVM)) does not satisfy the conditions derived in the step-by-step analysis for SGDClassifier with Squared Error Loss and L2 Penalty (correct choice: C). E: **Incorrect:** Option E (LassoClassifer) does not satisfy the conditions derived in the step-by-step analysis for SGDClassifier with Squared Error Loss and L2 Penalty (correct choice: C).

**Answer: B** **Topic:** Naive Bayes — *Returning Class Probabilities with predict_proba in GaussianNB* ### Step-by-Step Solution **Step 1:** In `scikit-learn` classifiers (including `GaussianNB`), compare the prediction methods: - `gnb.predict(X_test)` returns only the discrete predicted class label for each test sample. - `gnb.predict_proba(X_test)` **directly returns the posterior probability of each class for every test sample** (an array of shape `(n_samples, n_classes)` where each row sums to $1$). - `gnb.score(X_test, y_test)` returns the overall mean accuracy on the test set. **Step 2:** Therefore, **B** (`Calling gnb.predict_proba(X_test) to directly get the probabilities of each class for the test samples`) is correct. **Key Takeaway / Tip:** Always use `.predict_proba(X)` when you need continuous class probability estimates rather than hard class labels. A: **Incorrect:** Option A (

) does not satisfy the conditions derived in the step-by-step analysis for Returning Class Probabilities with predict_proba in GaussianNB (correct choice: B). B: **Correct:** Option B (

) matches our derived result under Returning Class Probabilities with predict_proba in GaussianNB. Always use `.predict_proba(X)` when you need continuous class probability estimates rather than hard class labels. C: **Incorrect:** Option C (

) does not satisfy the conditions derived in the step-by-step analysis for Returning Class Probabilities with predict_proba in GaussianNB (correct choice: B). D: **Incorrect:** Option D (

) does not satisfy the conditions derived in the step-by-step analysis for Returning Class Probabilities with predict_proba in GaussianNB (correct choice: B).

**Answer: A** **Topic:** Support Vector Machines — *Effect of Increasing C on One-vs-Rest SVM Decision Boundaries* ### Step-by-Step Solution **Step 1:** In `sklearn.svm.SVC`, the hyperparameter $C$ controls the penalty for misclassified or margin-violating training points. **Step 2:** As $C$ **increases**, the optimizer penalizes training errors more heavily and narrows the margin to classify as many training points correctly as possible. **Step 3:** Consequently, **the decision boundaries become more complex and tightly fitted to the training data** (**A**). **Key Takeaway / Tip:** Increasing $C$ reduces regularization, making the SVM decision boundary wigglier and more tightly fitted to individual training points. A: **Correct:** Option A (The boundaries become more complex and tightly fitted to the training data.) matches our derived result under Effect of Increasing C on One-vs-Rest SVM Decision Boundaries. Increasing $C$ reduces regularization, making the SVM decision boundary wigglier and more tightly fitted to individual training points. B: **Incorrect:** Option B (The boundaries become more regular and smooth.) does not satisfy the conditions derived in the step-by-step analysis for Effect of Increasing C on One-vs-Rest SVM Decision Boundaries (correct choice: A). C: **Incorrect:** Option C (Decision boundaries are unaffected by C.) does not satisfy the conditions derived in the step-by-step analysis for Effect of Increasing C on One-vs-Rest SVM Decision Boundaries (correct choice: A). D: **Incorrect:** Option D (The number of support vectors decreases.) does not satisfy the conditions derived in the step-by-step analysis for Effect of Increasing C on One-vs-Rest SVM Decision Boundaries (correct choice: A).

**Answer: C** **Topic:** Regression Metrics — *Computing R2 Score from True and Predicted Values* ### Step-by-Step Solution **Step 1:** Given `y_true = [3, -0.5, 2, 7]` and `y_pred = [2.5, 0.0, 2, 8]`, compute the Residual Sum of Squares ($SS_{\text{res}} = \sum (y_i - \hat{y}_i)^2$): - Residuals: $(3 - 2.5) = 0.5$, $(-0.5 - 0.0) = -0.5$, $(2 - 2) = 0$, $(7 - 8) = -1$ - $SS_{\text{res}} = (0.5)^2 + (-0.5)^2 + 0^2 + (-1)^2 = 0.25 + 0.25 + 0 + 1.0 = 1.5$ **Step 2:** Compute the mean $\bar{y}$ and Total Sum of Squares ($SS_{\text{tot}} = \sum (y_i - \bar{y})^2$): - $\bar{y} = \frac{3 - 0.5 + 2 + 7}{4} = \frac{11.5}{4} = 2.875$ - $SS_{\text{tot}} = (3 - 2.875)^2 + (-0.5 - 2.875)^2 + (2 - 2.875)^2 + (7 - 2.875)^2 = 0.015625 + 11.390625 + 0.765625 + 17.015625 = 29.1875$ **Step 3:** Calculate $R^2$: $$R^2 = 1 - \frac{SS_{\text{res}}}{SS_{\text{tot}}} = 1 - \frac{1.5}{29.1875} \approx 1 - 0.05139 = 0.9486 \approx 0.95$$ which matches **C**. **Key Takeaway / Tip:** This is the classic example from the official `sklearn.metrics.r2_score` documentation, which evaluates to `0.9486` ($\approx 0.95$). A: **Incorrect:** Option A (0.99) does not satisfy the conditions derived in the step-by-step analysis for Computing R2 Score from True and Predicted Values (correct choice: C). B: **Incorrect:** Option B (0.87) does not satisfy the conditions derived in the step-by-step analysis for Computing R2 Score from True and Predicted Values (correct choice: C). C: **Correct:** Option C (0.95) matches our derived result under Computing R2 Score from True and Predicted Values. This is the classic example from the official `sklearn.metrics.r2_score` documentation, which evaluates to `0.9486` ($\approx 0.95$). D: **Incorrect:** Option D (0.80) does not satisfy the conditions derived in the step-by-step analysis for Computing R2 Score from True and Predicted Values (correct choice: C).

**Answer: A** **Topic:** Support Vector Machines — *Predicting with a Linear SVC on 2D Data* ### Step-by-Step Solution **Step 1:** Inspect the training data $X$ and labels $y$: - Class `0` points: `[-1, -1]`, `[0.5, -1.5]`, `[0, -1]`, `[0, 0]` (points with $x_1 + x_2 \le 0$) - Class `1` points: `[1, 1]`, `[1, 0]` (points with $x_1 \ge 1$ and $x_2 \ge 0$) **Step 2:** The test point is `[[1.5, 1]]`, which lies even further in the positive Class `1` region than both `[1, 1]` and `[1, 0]`. **Step 3:** Therefore, `clf.predict([[1.5, 1]])` outputs **`[1]`**, which matches **A**. **Key Takeaway / Tip:** Since `[1.5, 1]` is right next to the positive training point `[1, 1]` (class `1`), any linear separator puts `[1.5, 1]` in class `1`. A: **Correct:** Option A (

) matches our derived result under Predicting with a Linear SVC on 2D Data. Since `[1.5, 1]` is right next to the positive training point `[1, 1]` (class `1`), any linear separator puts `[1.5, 1]` in class `1`. B: **Incorrect:** Option B (

) does not satisfy the conditions derived in the step-by-step analysis for Predicting with a Linear SVC on 2D Data (correct choice: A). C: **Incorrect:** Option C (

) does not satisfy the conditions derived in the step-by-step analysis for Predicting with a Linear SVC on 2D Data (correct choice: A). D: **Incorrect:** Option D (

) does not satisfy the conditions derived in the step-by-step analysis for Predicting with a Linear SVC on 2D Data (correct choice: A).

**Answer: A** **Topic:** Linear Classifiers — *Purpose of tol Parameter in SGDClassifier* ### Step-by-Step Solution **Step 1:** In `SGDClassifier`, the `tol` parameter specifies the stopping criterion tolerance: training stops when the loss does not improve by at least `tol` for `n_iter_no_change` consecutive epochs. **Step 2:** Compare with the options: - **A**: 'It defines the tolerance for stopping criteria.' — **Correct**. - **B**: Learning rate schedule is controlled by `learning_rate`. - **C**: Regularization strength is controlled by `alpha`. - **D**: Number of iterations is controlled by `max_iter`. **Step 3:** Therefore, **A** is correct. **Key Takeaway / Tip:** In `scikit-learn` iterative solvers, `tol` always defines the convergence tolerance for the stopping criterion. A: **Correct:** Option A (It defines the tolerance for stopping criteria.) matches our derived result under Purpose of tol Parameter in SGDClassifier. In `scikit-learn` iterative solvers, `tol` always defines the convergence tolerance for the stopping criterion. B: **Incorrect:** Option B (It defines the learning rate schedule.) does not satisfy the conditions derived in the step-by-step analysis for Purpose of tol Parameter in SGDClassifier (correct choice: A). C: **Incorrect:** Option C (It controls regularization strength.) does not satisfy the conditions derived in the step-by-step analysis for Purpose of tol Parameter in SGDClassifier (correct choice: A). D: **Incorrect:** Option D (It controls the number of iterations.) does not satisfy the conditions derived in the step-by-step analysis for Purpose of tol Parameter in SGDClassifier (correct choice: A).

**Answer: A, C, D** **Topic:** Support Vector Machines — *Valid Built-in Kernels in sklearn.svm.SVC* ### Step-by-Step Solution **Step 1:** Recall the built-in string kernel names accepted by the `kernel` parameter of `sklearn.svm.SVC`: - `'linear'`, `'poly'`, `'rbf'`, `'sigmoid'`, and `'precomputed'`. **Step 2:** Check the given options: - **A (`'linear'`)**: Valid. - **B (`'quadratic'`)**: **Invalid** (a degree-2 polynomial kernel is specified using `kernel='poly', degree=2`, not `'quadratic'`). - **C (`'rbf'`)**: Valid. - **D (`'sigmoid'`)**: Valid. **Step 3:** Therefore, **A, C, and D** are valid kernels. **Key Takeaway / Tip:** `SVC` uses `kernel='poly'` (with `degree=2`) rather than `'quadratic'`. A: **Correct:** Option A (’linear’) matches our derived result under Valid Built-in Kernels in sklearn.svm.SVC. `SVC` uses `kernel='poly'` (with `degree=2`) rather than `'quadratic'`. B: **Incorrect:** Option B (’quadratic’) does not satisfy the conditions derived in the step-by-step analysis for Valid Built-in Kernels in sklearn.svm.SVC (correct choice: A, C, D). C: **Correct:** Option C (’rbf’) matches our derived result under Valid Built-in Kernels in sklearn.svm.SVC. `SVC` uses `kernel='poly'` (with `degree=2`) rather than `'quadratic'`. D: **Correct:** Option D (’sigmoid’) matches our derived result under Valid Built-in Kernels in sklearn.svm.SVC. `SVC` uses `kernel='poly'` (with `degree=2`) rather than `'quadratic'`.

**Answer: B, C, D** **Topic:** Logistic Regression — *Mitigating Overfitting in LogisticRegression* ### Step-by-Step Solution **Step 1:** When a `LogisticRegression` model is **overfitting** the training data, we want to reduce model variance/complexity or provide more training data: - **A is false**: Increasing $C$ *reduces* regularization strength ($\lambda = 1/C$), which increases overfitting. - **B is true**: Setting `penalty='l2'` and **decreasing $C$** increases regularization strength, shrinking weights and reducing overfitting. - **C is true**: Removing irrelevant or redundant features via feature selection reduces the number of free parameters and mitigates overfitting. - **D is true**: Increasing the size of the training dataset helps the model generalize better and reduces overfitting. **Step 2:** Therefore, **B, C, and D** are correct. **Key Takeaway / Tip:** To combat overfitting in `LogisticRegression`, **decrease** $C$ (stronger regularization), reduce irrelevant features, or gather more training samples. A: **Incorrect:** Option A (

) does not satisfy the conditions derived in the step-by-step analysis for Mitigating Overfitting in LogisticRegression (correct choice: B, C, D). B: **Correct:** Option B (

) matches our derived result under Mitigating Overfitting in LogisticRegression. To combat overfitting in `LogisticRegression`, **decrease** $C$ (stronger regularization), reduce irrelevant features, or gather more training samples. C: **Correct:** Option C (

) matches our derived result under Mitigating Overfitting in LogisticRegression. To combat overfitting in `LogisticRegression`, **decrease** $C$ (stronger regularization), reduce irrelevant features, or gather more training samples. D: **Correct:** Option D (

) matches our derived result under Mitigating Overfitting in LogisticRegression. To combat overfitting in `LogisticRegression`, **decrease** $C$ (stronger regularization), reduce irrelevant features, or gather more training samples.

**Answer: A, C, D** **Topic:** Pipelines and RidgeClassifier — *Analyzing a MinMaxScaler + RidgeClassifier Pipeline* ### Step-by-Step Solution **Step 1:** Examine `make_pipeline(MinMaxScaler(), RidgeClassifier(alpha=1.0, fit_intercept=True))`: - **A is true**: `RidgeClassifier` applies $L_2$ (Tikhonov) regularization to prevent overfitting. - **B is false**: `fit_intercept=True` means the model **will** include an intercept (bias) term. - **C is true**: When `pipeline.fit(X, y)` is called, the first step (`MinMaxScaler()`) scales the features to $[0, 1]$ before passing them to `RidgeClassifier`. - **D is true**: The `alpha` parameter controls the strength of the $L_2$ regularization penalty. **Step 2:** Therefore, **A, C, and D** are correct. **Key Takeaway / Tip:** `fit_intercept=True` enables the intercept term; setting it to `False` omits the intercept. A: **Correct:** Option A (

) matches our derived result under Analyzing a MinMaxScaler + RidgeClassifier Pipeline. `fit_intercept=True` enables the intercept term; setting it to `False` omits the intercept. B: **Incorrect:** Option B (

) does not satisfy the conditions derived in the step-by-step analysis for Analyzing a MinMaxScaler + RidgeClassifier Pipeline (correct choice: A, C, D). C: **Correct:** Option C (

) matches our derived result under Analyzing a MinMaxScaler + RidgeClassifier Pipeline. `fit_intercept=True` enables the intercept term; setting it to `False` omits the intercept. D: **Correct:** Option D (

) matches our derived result under Analyzing a MinMaxScaler + RidgeClassifier Pipeline. `fit_intercept=True` enables the intercept term; setting it to `False` omits the intercept.

**Answer: A, B** **Topic:** Logistic Regression — *Choosing Solvers for Large-Scale Datasets* ### Step-by-Step Solution **Step 1:** In `sklearn.linear_model.LogisticRegression`, different optimization `solver` choices scale differently with dataset size: - **`'sag'` (Stochastic Average Gradient, A)** and **`'saga'` (B)** use stochastic gradient updates whose per-iteration cost is independent of $n$, making them specifically designed and fastest for **large datasets** (millions of records with many features). - **`'lbfgs'` (C)** and **`'liblinear'` (D)** are batch/coordinate-descent methods that are slower on massive datasets (`'liblinear'` is best for small-to-medium datasets). **Step 2:** Therefore, **A and B** (`sag` and `saga`) are the most suitable choices. **Key Takeaway / Tip:** According to the `scikit-learn` documentation, `'sag'` and `'saga'` are the recommended solvers for large datasets because they use stochastic average gradient descent. A: **Correct:** Option A (

) matches our derived result under Choosing Solvers for Large-Scale Datasets. According to the `scikit-learn` documentation, `'sag'` and `'saga'` are the recommended solvers for large datasets because they use stochastic average gradient descent. B: **Correct:** Option B (

) matches our derived result under Choosing Solvers for Large-Scale Datasets. According to the `scikit-learn` documentation, `'sag'` and `'saga'` are the recommended solvers for large datasets because they use stochastic average gradient descent. C: **Incorrect:** Option C (

) does not satisfy the conditions derived in the step-by-step analysis for Choosing Solvers for Large-Scale Datasets (correct choice: A, B). D: **Incorrect:** Option D (

) does not satisfy the conditions derived in the step-by-step analysis for Choosing Solvers for Large-Scale Datasets (correct choice: A, B).

**Answer: 0.0175** **Topic:** Regression Metrics — *Calculating Mean Squared Error (MSE)* ### Step-by-Step Solution **Step 1:** Compute the residuals $e_i = y_{\text{true}, i} - y_{\text{pred}, i}$ for the $n = 4$ samples: - $e_1 = 1.2 - 1.0 = 0.2 \implies e_1^2 = 0.04$ - $e_2 = 2.4 - 2.5 = -0.1 \implies e_2^2 = 0.01$ - $e_3 = 3.5 - 3.6 = -0.1 \implies e_3^2 = 0.01$ - $e_4 = 4.6 - 4.5 = 0.1 \implies e_4^2 = 0.01$ **Step 2:** Take the average of the squared errors: $$\text{MSE} = \frac{1}{4}(0.04 + 0.01 + 0.01 + 0.01) = \frac{0.07}{4} = 0.0175$$ **Key Takeaway / Tip:** Always average the squared residuals ($0.07 / 4 = 0.0175$, which lies right in the middle of the accepted range `0.012 to 0.022`).

**Answer: 60** **Topic:** Nearest Neighbors — *KNeighborsRegressor with k Equal to Training Set Size* ### Step-by-Step Solution **Step 1:** Notice that `n_neighbors=len(y_train)` sets $k = 11$, which is the **entire training set** of 11 points, with `weights='uniform'`. **Step 2:** When $k$ equals the total number of training samples and weights are uniform, `KNeighborsRegressor` simply predicts the **mean of all training targets** `y_train` for any test point: - `y_train = [10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110]` **Step 3:** Compute the average of this arithmetic progression of 11 numbers: $$\hat{y} = \frac{10 + 110}{2} = 60$$ **Key Takeaway / Tip:** Whenever `n_neighbors = len(y_train)` with `weights='uniform'`, there is no need to compute distances—every training point is a neighbor, so the prediction is just `np.mean(y_train) = 60`!

**Answer: 7** **Topic:** Feature Extraction — *Counting Unique Bigrams with CountVectorizer* ### Step-by-Step Solution **Step 1:** `CountVectorizer(ngram_range=(2, 2))` lowercases each document, tokenizes words of length $\ge 2$, and extracts all consecutive 2-word pairs (**bigrams**) within each document separately: **Step 2:** Extract the bigrams from each of the two documents: - **Document 1 (`'The quick brown fox'`)**: 4 words $\implies 4 - 1 = 3$ bigrams: 1. `'the quick'` 2. `'quick brown'` 3. `'brown fox'` - **Document 2 (`'jumps over the lazy dog'`)**: 5 words $\implies 5 - 1 = 4$ bigrams: 4. `'jumps over'` 5. `'over the'` 6. `'the lazy'` 7. `'lazy dog'` **Step 3:** Since all $3 + 4 = 7$ bigrams are distinct, `len(bigrams)` is **7**. **Key Takeaway / Tip:** A document with $m$ words produces $m - 1$ bigrams (and bigrams never cross document boundaries), giving $(4 - 1) + (5 - 1) = 3 + 4 = 7$ unique bigrams.