Questions and published explanations below are available without starting a test. Some questions may not have a published solution yet.
9 / 11 published solutions
Question 2 MCQ · 4.0 marks
Consider a binary classification problem in
. The feature space is partitioned into the following
four regions:
Each region contains the same number of observations. All observations in regions
and
belong to the positive class, while all observations in regions
and
belong to the negative
class.
A decision tree is constructed using axis-aligned binary splits of the form
where
is any real number. At each node, the split is chosen to maximize the information gain
based on entropy.
What is the minimum number of binary questions (splits) that the decision tree must ask to obtain
leaf nodes with zero entropy?
Suppose an email is represented by a binary feature vector
A generative classifier first chooses the class label
according to the prior distribution
. It
then generates the feature vector
conditioned on the chosen class. The classifier assumes that,
given the class label, the features are conditionally independent.
Which of the following expressions correctly represents
**1. Apply Bayes’ theorem:** The desired posterior is \[P(Y=\mathrm{Spam}\mid X=x)=\frac{P(X=x\mid Y=\mathrm{Spam})P(Y=\mathrm{Spam})}{P(X=x)}.\]
**2. Use conditional, not marginal, independence:** The model assumes that features are independent given Y. Hence the likelihood factors as \(P(X=x\mid Y=\mathrm{Spam})=\prod_{i=1}^dP(X_i=x_i\mid Y=\mathrm{Spam})\).
**3. Substitute the factorization:** Multiplying these conditional feature probabilities by the class prior and dividing by the evidence gives option D. The evidence sums the joint score over all possible classes and makes the posterior probabilities sum to one.
**Answer: D — the product of class-conditional feature probabilities times the spam prior, divided by P(X=x).**
A: **Incorrect:** This omits the likelihood of the observed feature vector within the spam class.
B: **Incorrect:** Conditional independence does not replace class-conditional feature probabilities with unconditional marginals.
C: **Incorrect:** A product of featurewise posteriors is not Bayes’ rule and miscounts prior/evidence factors.
D: **Correct:** This is the class prior times the factored conditional likelihood, normalized by the evidence.
Under the probabilistic view of linear regression, assume
where the observations are independent.
Suppose the true noise variance is
instead of
, while the dataset and the observed labels
remain unchanged.
What happens to the maximum likelihood estimator
6406537086659.
It is unchanged.
6406537086660.
It is scaled by
.
6406537086661.
It is scaled by
.
6406537086662.
It cannot be determined without knowing
.
Published solution
**1. Write the negative log-likelihood:** With common variance \(s^2>0\), it is \(\tfrac n2\log(2\pi s^2)+\tfrac1{2s^2}\|y-Xw\|^2\). The first term is constant in w and the second is a positive multiple of the same squared-error objective.
**2. Change the variance:** Replacing \(\sigma^2\) by \(4\sigma^2\) changes the multiplier from \(1/(2\sigma^2)\) to \(1/(8\sigma^2)\). Multiplication by a positive constant does not change the minimizing weights.
**3. Recover the normal equation:** Differentiating still gives \(X^TX\widehat w=X^Ty\), and, when invertible, \(\widehat w=(X^TX)^{-1}X^Ty\). The estimator is unchanged for the same observed data, although its statistical uncertainty depends on the noise variance.
**Answer: A — the maximum-likelihood weight estimator is unchanged.**
**Convention note:** The source displays the alternative feature-by-sample convention (XXᵀ)⁻¹Xy. Both orientations express the same normal-equation argument; the inverse formula assumes full feature rank.
A: **Correct:** The least-squares minimizer is unaffected by a common positive scaling of the loss.
B: **Incorrect:** Four times the variance does not multiply the fitted weights by four.
C: **Incorrect:** The objective’s scale changes, not the location of its minimum.
D: **Incorrect:** The common variance cancels from the normal equations.
On a machine that folds plastic film, the operating temperature can be varied between
and
. To study the relationship between temperature and fold thickness,
observations
of temperature and the corresponding fold thickness were collected. The scatter plot along with
the fitted simple linear regression line is shown below.
Determine, by inspecting the figure, which of the following sets of parameter estimates for the
simple linear regression model is most appropriate.
**Note:** The regression model is
where
Here,
•
is the estimated intercept,
•
is the estimated slope, and
•
is the estimated standard deviation of the error term.
**1. Estimate the slope from the plotted line:** The fitted thickness drops from roughly 135 near temperature 130 to roughly 86 near temperature 185. Thus the slope is about \((86-135)/(185-130)=-49/55\approx-0.9\). A positive slope is inconsistent with the figure.
**2. Recover the intercept:** Using \(y\approx\beta_0-0.9x\) near (130,135) gives \(\beta_0\approx135+0.9(130)=252\). The intercept is a value extrapolated to temperature zero, not the plotted starting thickness.
**3. Compare residual scales:** The observations lie close to the line; their vertical deviations are small relative to the 50-unit displayed thickness range. A standard deviation of 360 is implausibly large, whereas 3.6 is the only small proposed alternative.
**4. Choose the closest listed parameter set:** The combination is \(\widehat\beta_0=252,\widehat\beta_1=-0.9,\widehat\sigma=3.6\). The exact residual standard deviation cannot be calculated from the image alone; this is an inspection-based choice.
**Answer: C — β₀=252, β₁=−0.9, σ=3.6.**
A: **Incorrect:** Zero intercept and residual scale 360 are both inconsistent with the plotted line.
B: **Incorrect:** The plotted slope is negative, not positive; zero intercept is also wrong.
C: **Correct:** The slope, extrapolated intercept and comparatively small error scale fit the figure best.
D: **Incorrect:** A negative intercept would place the entire displayed line far below the plotted observations.
E: **Incorrect:** The line coefficients fit, but a noise standard deviation of 360 does not.
**1. Derive the likelihood contribution:** The observation noise variance is 1. Thus, up to a constant, the negative log-likelihood is \(\tfrac12\sum_i(w^Tx_i-y_i)^2\).
**2. Derive the Gaussian-prior penalty:** For \(w\sim\mathcal N(0,\sigma^2I)\), \(-\log p(w)=\tfrac1{2\sigma^2}\|w\|^2+\text{constant}\). MAP minimizes the sum of the negative log-likelihood and negative log-prior.
**3. Match the ridge coefficient:** The stated ridge objective has penalty \(\tfrac\lambda2\|w\|^2\). Equating \(\lambda/2=1/(2\sigma^2)\) gives \(\lambda=1/\sigma^2\). A more concentrated prior therefore produces stronger regularization.
**Answer: D — \(\lambda=1/\sigma^2\).**
A: **Incorrect:** The penalty strength is inversely proportional to prior variance, not equal to it.
B: **Incorrect:** This misses a factor of two when compared with the stated λ/2 penalty.
C: **Incorrect:** This works only in the special case σ²=1, not for every positive prior variance.
D: **Correct:** The negative log-prior coefficient gives λ=1/σ² exactly.
A node in a decision tree contains an equal number of positive and negative training examples.
What is the entropy of this node?
6406537086655.
0
6406537086656.
0.5
6406537086657.
1
6406537086658.
It depends on the total number of training examples.
Published solution
**1. Convert the equal counts to probabilities:** Since the node contains equally many positive and negative examples, \(p_+=p_-=1/2\).
**2. Evaluate binary entropy:** \[H=-\tfrac12\log_2\tfrac12-\tfrac12\log_2\tfrac12=\tfrac12+\tfrac12=1\text{ bit}.\] This is the maximum entropy of a binary class distribution; it depends on proportions, not the total sample count.
**Answer: C — 1 bit.**
**Convention note:** The standard decision-tree convention uses logarithms to base 2. Natural-log entropy for the same distribution would be ln(2) nats.
A: **Incorrect:** Zero entropy requires a pure node containing only one class.
B: **Incorrect:** One half is each class probability, not the sum defining entropy.
C: **Correct:** A balanced binary distribution has entropy 1 bit.
D: **Incorrect:** Equal class proportions determine entropy regardless of the number of examples.
Both students use gradient descent with the same learning rate
and the same initial parameter
vector
.
Which one of the following statements is correct?
6406537086663.
They converge to different minimizers, because the losses differ.
6406537086664.
They converge to the same minimizer, but
's steps are
times smaller;
would need learning rate
to reproduce
's trajectory.
6406537086665.
They converge to the same minimizer, and their iterates coincide at every step.
6406537086666.
They converge to the same minimizer, but
's steps are
times larger.
Published solution
**1. Relate the objectives:** Using the intended mean-squared-error definition in the prose, the losses satisfy \(L_Q(w)=L_P(w)/n\). They have the same minimizer set because n is a positive constant, but their gradients satisfy \(\nabla L_Q=\nabla L_P/n\).
**2. Compare updates at the same parameter:** With the same rate η, P updates by \(-\eta\nabla L_P\), while Q updates by \(-\eta\nabla L_P/n\). Q’s update at that location is n times smaller.
**3. Match entire trajectories:** Give Q learning rate nη. Then \(-(n\eta)\nabla L_Q=-\eta\nabla L_P\), so identical initialization produces identical iterates at every step by induction.
**4. State the convergence qualification:** With the original same rate, convergence requires a rate stable for the relevant objective. If both converge from the same initialization on this quadratic, they reach the same solution, including the same unchanged null-space component in a rank-deficient problem.
**Answer: B — Q’s updates are n times smaller; Q needs rate nη to match P.**
**Convention note:** The printed MSE formula visibly omits the minus sign between wᵀx_i and y_i, showing their product instead. The solution uses the prose’s intended MSE, (1/(2n)) sum_i (wᵀx_i−y_i)²; the literal printed product is a different objective and does not support the advertised SSE/MSE comparison. The option’s statement that they converge also assumes an appropriately small learning rate. The prompt alone does not guarantee stability for an arbitrary η; also, update-size comparisons apply at a common parameter, since the unrescaled trajectories generally separate.
A: **Incorrect:** Positive scaling changes gradients but not the minimizing set.
B: **Correct:** Q has a gradient smaller by n and needs nη to reproduce P’s updates.
C: **Incorrect:** The same rate generally gives different iterates unless n=1 or the gradient vanishes.
D: **Incorrect:** Q’s steps at a common parameter are smaller, not larger.
**1. Express the possible prediction vectors:** The feature columns across the three samples are \(f_1=(1,0,1)^T\) and \(f_2=(0,1,1)^T\). A no-intercept linear model predicts \(w_1f_1+w_2f_2=(w_1,w_2,w_1+w_2)^T\).
**2. Derive the exact-fit condition:** The first two labels fix \(w_1=y_1\) and \(w_2=y_2\). Zero squared training error then requires \(y_3=y_1+y_2\).
**3. Check the proposed labels:** For (1,2,3), the condition is 3=1+2 and weights (1,2) fit exactly. The other third coordinates are 4 versus 1+2, 4 versus 2+1, and 1 versus 0+0; all violate the necessary condition.
**Answer: A — \(y=(1,2,3)^T\).**
A: **Correct:** Weights (1,2) produce exactly (1,2,3).
B: **Incorrect:** The third prediction must be 3, not 4.
C: **Incorrect:** The first two coordinates force the third prediction to 3, not 4.
D: **Incorrect:** Zero first and second predictions force the third prediction to zero.
is the optimal weight obtained after minimizing the objective function of linear
regression. Which statement is correct?
6406537086680.
and the training error is
.
6406537086681.
and the squared training error equals
.
6406537086682.
and the squared training error equals
.
6406537086683.
The normal equations have no solution.
Published solution
**1. Describe the model’s prediction subspace:** Write Z for the usual sample-by-feature design matrix; in the paper’s orientation Z=Xᵀ. Predictions Zw lie in the span of the feature vectors, and the assumption is \(Z^Ty=0\).
**2. Expand squared error using orthogonality:** \[\|y-Zw\|^2=\|y\|^2-2y^TZw+\|Zw\|^2=\|y\|^2+\|Zw\|^2.\] The cross term is zero for every w.
**3. Minimize the remaining term:** The smallest possible prediction norm is zero, attained by w=0. Every optimal weight vector therefore satisfies \(Zw^*=0\), and its squared training error is \(\|y\|^2\), which is positive because y is nonzero.
**4. Avoid assuming unique weights:** If Z has a nontrivial null space, there can be several optimal weight vectors. Their predictions are nevertheless all zero, which is the statement the correct option makes.
**Answer: B — \(X^Tw^*=0\), with squared training error \(\|y\|_2^2\).**
A: **Incorrect:** A nonzero vector orthogonal to the prediction space cannot also be fitted exactly within it.
B: **Correct:** The projection of y onto the feature span is zero, leaving squared residual norm ||y||².
C: **Incorrect:** With zero prediction and nonzero y, the squared error is positive.
D: **Incorrect:** The normal equations have at least the solution w=0.
A generative classifier models the joint distribution of the feature vector
and the class label
.
For a particular email represented by the feature vector
, suppose
and
Compute the posterior probability
Published solution
**1. Calculate the evidence from the two joint probabilities:** Spam and Not Spam are the two mutually exclusive classes. Thus \(P(X=x)=P(X=x,Y=Spam)+P(X=x,Y=Not\ Spam)=0.015+0.035=0.05\).
**2. Normalize the spam joint probability:** \[P(Y=Spam\mid X=x)=\frac{0.015}{0.05}=0.3.\]
**3. Distinguish joint probability from likelihood:** The supplied numbers already include their class priors, so multiplying by a prior again would double-count it. As a check, the other posterior is 0.035/0.05=0.7 and the two sum to 1.
**Answer: 0.3.**
