**1. Write the conditional density:** The model \(Y=wX+\epsilon\), with independent \(\epsilon\sim\mathcal N(0,\sigma^2)\), gives \(p(y_i\mid x_i,w)=(2\pi\sigma^2)^{-1/2}\exp[-(y_i-wx_i)^2/(2\sigma^2)]\). Independence makes the likelihood a product of these densities.
**2. Remove constants that do not depend on w:** The positive factor \((2\pi\sigma^2)^{-n/2}\) does not affect the maximizing w, so maximizing the product of exponential terms is equivalent to maximizing the complete likelihood.
**3. Take logarithms and change the sign:** The log-likelihood is \(-\tfrac n2\log(2\pi\sigma^2)-\tfrac1{2\sigma^2}\sum_i(y_i-wx_i)^2\). Thus maximizing it is equivalent to minimizing the scaled squared error, not maximizing that error.
**Answer: A, B, D — the full likelihood, the likelihood without its constant factor, and the squared-error minimization.**
A: **Correct:** This is the product of all independent Gaussian conditional densities.
B: **Correct:** Only a positive factor independent of w has been dropped.
C: **Incorrect:** Maximizing positive squared error has the opposite direction from maximum likelihood.
D: **Correct:** Minimizing negative log-likelihood gives precisely this squared-error objective.
E: **Incorrect:** C is not equivalent to maximum likelihood, so “All” is false.
**1. Reduce maximum likelihood to least squares:** With known positive noise variance, minimize \(S(w)=\sum_i(y_i-wx_i)^2\). No intercept is present in the specified model.
**2. Differentiate and solve:** \[S^\prime(w)=-2\sum_i x_i(y_i-wx_i)=0\quad\Longrightarrow\quad\widehat w=\frac{\sum_i x_iy_i}{\sum_i x_i^2}.\]
**3. Verify it is a minimum:** \(S^{\prime\prime}(w)=2\sum_i x_i^2>0\) whenever at least one input is nonzero. Under that condition the estimate is unique.
**Answer: A — \(\widehat w_{ML}=\frac{\sum_i x_iy_i}{\sum_i x_i^2}\).**
**Convention note:** The formula assumes at least one nonzero x_i. If every x_i=0, the likelihood is independent of w and the parameter is not identifiable.
A: **Correct:** This is the stationary-point solution of the stated no-intercept least-squares problem.
B: **Incorrect:** Ratios of sums do not solve the normal equation, which involves sums of products and squares.
C: **Incorrect:** This centered formula estimates a slope when a separate intercept is fitted, unlike the stated model.
D: **Incorrect:** The extra factor n is not produced by differentiation; averaging the loss cancels from both sides.
**1. Count observations in each class:** Of the six samples, rows 1 and 2 have label 0; rows 3 and 4 have label +1; rows 5 and 6 have label -1. Each class therefore appears twice.
**2. Estimate the class priors:** The multinomial maximum-likelihood estimate is class count divided by total count. Consequently \(\widehat p_1=\widehat p_2=\widehat p_3=2/6=1/3\). Feature independence is not needed for this prior calculation.
**Answer: B — \((1/3,1/3,1/3)\).**
A: **Incorrect:** These unequal priors would require unequal class counts.
B: **Correct:** Each label occurs in two of the six observations.
C: **Incorrect:** There are two observations of class 0 and two of class +1, not one and three.
D: **Incorrect:** Class +1 is not more frequent than either other class.
belong to? Assume that no Laplacian smoothing has been
done.
Published solution
**1. Use the Bernoulli Naive Bayes score:** For \(x=(1,1,1,1)^T\), the unnormalized class score is \(P(y=c)\prod_{j=1}^4P(f_j=1\mid y=c)\). All three priors are \(1/3\).
**2. Calculate each class likelihood:** Class 0 has feature-one probabilities \((1/2,1/2,1/2,1/2)\), giving score \((1/3)(1/2)^4=1/48\). Class +1 has probabilities \((1,1,1/2,0)\), giving zero. Class -1 has probabilities \((0,0,1/2,1)\), also giving zero.
**3. Normalize and classify:** The only positive score is that of class 0, so its posterior is 1 and the other two posteriors are 0. The predicted label is 0. Without smoothing, an unseen required feature within a class makes its likelihood zero.
**Answer: 0.**
**1. Form the design matrix:** The three samples are \((x_1,x_2,y)=(1,0,2),(0,1,3),(1,1,4)\), and the fitted model has no intercept. Thus \(X=\begin{pmatrix}1&0\\0&1\\1&1\end{pmatrix}\), \(y=(2,3,4)^T\).
**2. Solve the normal equations:** \(X^TX=\begin{pmatrix}2&1\\1&2\end{pmatrix}\) and \(X^Ty=(6,7)^T\). Therefore \(2w_1+w_2=6\), \(w_1+2w_2=7\), giving \(w_1=5/3\), \(w_2=8/3\).
**3. Compute every residual:** Predictions are \(5/3,8/3,13/3\). The observed-minus-predicted residuals are \(1/3,1/3,-1/3\), whose squared sum is \(1/3\).
**4. Average and round:** \(\operatorname{MSE}=(1/3)/3=1/9=0.111\ldots\), or 0.11 to two decimal places. Dividing by the number of samples is essential; the sum of squares alone is not MSE.
**Answer: 0.11.**
Consider the following trained decision tree learned for the dataset in Figure 1.
Find the training error of the decision tree for the given dataset. Enter the answer correct to one
decimal place.
Published solution
**1. Read the tree as intervals:** The root sends \(x<2.5\) to class 0. Otherwise, the second test sends \(x<3\) to class 1, and \(x\ge3\) to class 0.
**2. Route all five training samples:** For x values \(0.5,1.5,2.5,2.7,3.5\), the predictions are \(0,0,1,1,0\), respectively. The true labels are \(0,0,1,0,1\). Equality at 2.5 follows the root’s false branch because the comparison is strict.
**3. Count classification errors:** The samples at 2.7 and 3.5 are incorrect, while the other three are correct. Hence training error is \(2/5=0.4\), rather than the accuracy \(3/5\).
**Answer: 0.4.**
**1. Translate the observed email:** It contains words \(u_1,u_3\) and lacks \(u_2\), so \(X=(1,0,1)\). Use the absence probability for the second feature, not its listed presence probability.
**2. Compute both joint scores:** The spam prior is 0.4 and the ham prior is 0.6. Conditional independence gives \(s_{spam}=0.4(0.8)(1-0.7)(0.6)=0.0576\) and \(s_{ham}=0.6(0.1)(1-0.2)(0.3)=0.0144\).
**3. Apply Bayes normalization:** \[P(\theta=1\mid X)=\frac{0.0576}{0.0576+0.0144}=\frac{0.0576}{0.072}=0.80.\] A likelihood alone is not a posterior; the denominator includes both possible classes.
**Answer: 0.80.**
Consider a dataset of 1000 points for a classification problem using k-NN algorithm. Select the
correct statements from the following:
6406535937431.
If k = 5, it is enough if we store any 5 points in the training dataset.
6406535937432.
If k = 5, we need to store the entire dataset.
6406535937433.
The number of data-points that we have to store increases as k increases.
6406535937434.
The number of data-points that we have to store is independent of the value
of k.
Published solution
**1. Distinguish stored points from selected neighbors:** Standard exact k-NN retains the training feature vectors and labels. A query determines which points are closest, so an arbitrary fixed subset of five cannot replace the full set merely because k=5.
**2. Compare different k values:** The same 1000 stored samples support different choices of k. Changing k changes how many nearest labels vote at prediction time, not how many training observations the standard algorithm retains.
**Answer: B, D — retain the entire dataset; its storage size is independent of k.**
**Convention note:** This describes ordinary exact k-NN without a separately specified prototype reduction or approximate-neighbor scheme.
A: **Incorrect:** Any five training points need not be the five nearest to a future query.
B: **Correct:** The standard exact algorithm retains all 1000 training points to identify the query’s neighbors.
C: **Incorrect:** Increasing k changes the vote size, while the stored training set remains the same.
D: **Correct:** For the stated standard k-NN procedure, storage of the training set is independent of k.
**1. Locate the class switch:** The prediction changes when the two class posterior probabilities tie. Thus the decision boundary is \(\{x:P(Y=1\mid x)=P(Y=0\mid x)\}\), usually where each posterior equals 1/2.
**2. Substitute Bayes’ rule:** On inputs with positive evidence \(P(X=x)\), the two posterior expressions share the same denominator, which cancels from the equality.
**3. Apply Naive Bayes factorization:** Conditional independence gives \[P(Y=1)\prod_iP(X_i=x_i\mid Y=1)=P(Y=0)\prod_iP(X_i=x_i\mid Y=0).\] This equality describes the same boundary in terms of the unnormalized class scores.
**Answer: C, D — equal posteriors, or equivalently equal prior-times-likelihood products.**
A: **Incorrect:** A zero class-1 posterior generally places the point in the class-0 region, not at a tie.
B: **Incorrect:** A zero class-0 posterior generally places the point in the class-1 region, not at a tie.
C: **Correct:** The equality is exactly the posterior comparison threshold.
D: **Correct:** This follows by cancelling the common Bayes denominator and factoring each likelihood.
is unknown, we
would like to estimate it using linear regression. Adding priors to the parameter
, i.e.
is equivalent to adding a
penalty on the parameters in the objective function
defined by the log-likelihood.
6406535937443.
True
6406535937444.
False
Published solution
**1. Write the Gaussian likelihood loss:** For independent Gaussian observation noise, the negative log-likelihood contributes \(\tfrac1{2\sigma^2}\sum_i(y_i-w^Tx_i)^2\), up to constants.
**2. Take the negative log of the prior:** Independent priors \(w_j\sim\mathcal N(0,\lambda^2)\) contribute \(-\log p(w)=\tfrac1{2\lambda^2}\sum_jw_j^2+\text{constant}\).
**3. Combine for MAP estimation:** Minimizing the negative log-posterior therefore adds a squared L2 penalty to least squares. After multiplying by \(2\sigma^2\), its coefficient is \(\sigma^2/\lambda^2\); smaller prior variance means stronger shrinkage.
**Answer: A — True.**
**Convention note:** The usual equivalence assumes independent Gaussian coefficient priors, or an isotropic multivariate Gaussian prior. Here lambda squared denotes prior variance; it is not itself the ridge penalty coefficient.
A: **Correct:** A zero-mean Gaussian prior yields a quadratic L2 penalty in the MAP objective.
B: **Incorrect:** The negative log of this prior is quadratic, so the stated equivalence is true.
Consider four regression models (A, B, C, and D) fitted on the same dataset using Ridge or Lasso
regression with different regularization parameters.
Match each model with its corresponding regularization description given below.
**Models:**
A. Ridge regression with
B. Ridge regression with
C. Lasso regression with
D. Ridge regression with
**Regularization Effects:**
1. No regularization — same as Ordinary Least Squares (OLS)
2. Strong shrinkage — coefficients are small but non-zero
3. Moderate shrinkage — coefficients are reduced but not zero
4. Some coefficients are exactly zero due to sparsity.
Which of the following correctly matches the models with their effects?
6406535937448.
A–1, B–3, C–4, D–2
6406535937449.
A–1, B–2, C–4, D–3
6406535937450.
A–1, B–3, C–2, D–4
6406535937451.
A–2, B–1, C–3, D–4
Published solution
**1. Match the unpenalized model:** Ridge uses a squared L2 penalty. With \(\lambda=0\), model A reduces to ordinary least squares, so A maps to description 1.
**2. Compare ridge strengths:** Relative to \(\lambda=5\) in B, \(\lambda=10\) in D applies a stronger quadratic penalty. Thus B corresponds to moderate shrinkage (3) and D to stronger shrinkage (2).
**3. Identify the sparsity-inducing penalty:** Lasso uses an L1 penalty, whose corner at zero can make fitted coefficients exactly zero. Thus C maps to 4, giving A–1, B–3, C–4, D–2.
**Answer: A — A–1, B–3, C–4, D–2.**
**Convention note:** These are the intended characteristic effects. Ridge coefficients can equal zero for special datasets, and lasso need not set a coefficient to zero for every dataset or parameter choice; the descriptions are tendencies, not universal nonzero/zero guarantees.
A: **Correct:** All four descriptions match their penalty types and relative strengths.
B: **Incorrect:** This reverses moderate and strong shrinkage for the two ridge models.
C: **Incorrect:** This assigns the sparsity description to ridge rather than lasso.
D: **Incorrect:** Zero-penalty ridge is OLS, and the other assignments also mismatch the models.
**DLGenAI**
**Section Id :** 640653122508
**Section Number :** 9
**Section type :** Online
**Mandatory or Optional :** Mandatory
**Number of Questions :** 26
**Number of Questions to be attempted :** 26
**Section Marks :** 50
**Display Number Panel :** Yes
**Section Negative Marks :** 0
**Group All Questions :** No
**Enable Mark as Answered Mark for Review and**
No
**Clear Response :**
**Section Maximum Duration :** 0
**Section Minimum Duration :** 0
**Section Time In :** Minutes
**Maximum Instruction Time :** 0
Published solution
**1. Write the least-squares objective:** For the stated scalar slope, \(L(w)=\tfrac1n\sum_i(y_i-wx_i)^2\). At any unconstrained least-squares minimizer its derivative is zero.
**2. Differentiate:** \[L^\prime(w)=-\frac2n\sum_i x_i(y_i-wx_i),\qquad\frac1n\sum_i(y_i-w^*x_i)x_i=0.\] This is the normal equation: the residual vector is orthogonal to the input column.
**3. Distinguish other orthogonality claims:** Residuals are not generally orthogonal to the response vector or to arbitrary shifted quantities. For example, with inputs (1,2) and responses (1,0), the optimum is 0.2 and residuals are (0.8,-0.4); their dot product with x is zero but with y is 0.8.
**Answer: A — \(\frac1n\sum_i(y_i-w^*x_i)x_i=0\).**
A: **Correct:** This is exactly the zero-gradient condition for the slope.
B: **Incorrect:** The residual-response product is generally positive rather than zero, as the example shows.
C: **Incorrect:** An arbitrary response-centered quantity does not replace the input in the normal equation.
D: **Incorrect:** Without an intercept, the residual sum need not vanish, so shifting every input by an arbitrary x does not preserve the zero product.
