Questions and published explanations below are available without starting a test. Some questions may not have a published solution yet.
14 / 16 published solutions
Question 2 MSQ · 3.0 marks
Which among the following is true for the EM algorithm?
6406536642374.
At any iteration of the EM algorithm, the value of the objective function never
decreases.
6406536642375.
It optimizes an upper bound of the objective function.
6406536642376.
It optimizes a lower bound of the objective function.
6406536642377.
The EM algorithm is guaranteed to give a global solution.
6406536642378.
Upon convergence of the EM algorithm for a Gaussian Mixture Model, the
mixture proportions
become equal to the posterior probabilities
for all training
sample
.
Published solution
**1. Explain the lower bound:** EM introduces a latent posterior and uses a Jensen/variational lower bound on observed-data log likelihood. The E-step sets the latent distribution to the current exact posterior, making the bound tight there.
**2. Improve the bound:** An exact M-step maximizes this bound over parameters. Under ordinary exact-EM assumptions, the observed-data likelihood does not decrease between iterations. Numerical/approximate variants require their own qualifications.
**3. State limitations:** EM need not find a global optimum. In a GMM, mixture proportions update to averages of responsibilities: \(\pi_j=(1/n)\sum_i\gamma_{ij}\), not every example’s individual posterior probability.
**Answer: A, C — Selected: At any iteration of the EM algorithm, the value of the objective function never decreases.; It optimizes a lower bound of the objective function.**
A: **Correct:** Exact EM has a nondecreasing observed-likelihood property.
B: **Incorrect:** EM uses a lower bound, not an upper bound, for this maximization formulation.
C: **Correct:** The variational/Jensen lower bound is the optimized surrogate.
D: **Incorrect:** Local maxima and other stationary behavior are possible.
E: **Incorrect:** A mixture weight is the average responsibility, generally different from each individual responsibility.
is a 2-component Gaussian mixture distribution with density
where
, and
is known.
Based on the above data, answer the given subquestions.
Find the number of parameters to be estimated in the model.
Published solution
**1. Read what is known:** The two univariate Gaussian components share a specified known variance. The unknown means are mu1 and mu2.
**2. Count independent mixing weights:** Two weights sum to one, so one free weight determines the other.
**3. Add free parameters:** Two means plus one independent mixture weight give \(2+1=3\) parameters. Known variance is not estimated.
**Answer: 3.**
is a 2-component Gaussian mixture distribution with density
where
, and
is known.
Based on the above data, answer the given subquestions.
Suppose in the M-step of the EM algorithm, we have the following responsibilities for data points
and
:
where
Calculate the updated mean
for the second component. Enter the
answer correct to two decimal places.
Published solution
**1. Use the weighted-mean M-step:** For component j, \(\mu_j'=\sum_i\gamma_{ij}x_i/\sum_i\gamma_{ij}\).
**2. Select component-two responsibilities:** For x1=5 and x2=15, they are 0.1 and 0.8. Thus the numerator is \(0.1(5)+0.8(15)=0.5+12=12.5\), and the denominator is 0.9.
**3. Divide:** \(\mu_2'=12.5/0.9=13.888889\), rounded to 13.89. The result lies closer to 15 because that point carries most of this component’s responsibility.
**Answer: 13.89.**
**1. Use the closed-form forward diffusion:** \[x_t=\sqrt{\bar\alpha_t}x_0+\sqrt{1-\bar\alpha_t}\epsilon.\]
**2. Substitute the given values:** \(\bar\alpha_{200}=0.979\), x0=0.8 and epsilon=-1.2 give \(\sqrt{0.979}(0.8)+\sqrt{0.021}(-1.2)\).
**3. Evaluate:** The signal term is about 0.79156 and the noise term is about -0.17390, giving 0.61766. To two decimals, x200=0.62. Square roots scale signal/noise amplitudes, not their variances directly.
**Answer: 0.62.**
A factory uses a Variational Autoencoder (VAE) to automate the inspection of high-precision
ceramic tiles. The model is trained to identify defects by learning the distribution of ''normal" tiles.
For each input image
(resized to
pixels), the encoder
maps the image to a
dimensional latent space, while the decoder
attempts to reconstruct the original
pixels. The training objective to maximize the Evidence Lower Bound (ELBO), defined as:
A factory uses a Variational Autoencoder (VAE) to automate the inspection of high-precision
ceramic tiles. The model is trained to identify defects by learning the distribution of ''normal" tiles.
For each input image
(resized to
pixels), the encoder
maps the image to a
dimensional latent space, while the decoder
attempts to reconstruct the original
pixels. The training objective to maximize the Evidence Lower Bound (ELBO), defined as:
Assume the prior
is a standard normal distribution
.
Compute the latent regularization contribution
for this tile. Enter the answer correct to two
decimal places.
A published solution is not available for this question yet.
A factory uses a Variational Autoencoder (VAE) to automate the inspection of high-precision
ceramic tiles. The model is trained to identify defects by learning the distribution of ''normal" tiles.
For each input image
(resized to
pixels), the encoder
maps the image to a
dimensional latent space, while the decoder
attempts to reconstruct the original
pixels. The training objective to maximize the Evidence Lower Bound (ELBO), defined as:
Assume the prior
is a standard normal distribution
.
The total reconstruction mismatch ( negative log-likelihood ) for the training sample is found to be
. Calculate the ELBO objective value
for this sample. Round your answer to the
nearest integer.
Published solution
**1. Read the reconstruction term:** A negative log-likelihood mismatch of 186.4 means the expected log-likelihood contribution is -186.4.
**2. Compute KL from the supplied encoder:** With mu=(0.6,-0.4,0.2) and diagonal covariance \((e^{-0.8},e^{-1.2},e^{0.4})\),
\[KL=\frac12\sum_i[\mu_i^2+\sigma_i^2-1-\log\sigma_i^2]=\tfrac12[0.56+2.242348-3+1.6]=0.701174.\]
**3. Form the ELBO:** \(\mathcal L=-186.4-0.701174=-187.101174\), rounded to -187.
**Source note:** The separate source KL question’s range near 0.60 does not match the printed covariance; the correct KL above still yields this same rounded ELBO answer.
**Answer: -187.**
A factory uses a Variational Autoencoder (VAE) to automate the inspection of high-precision
ceramic tiles. The model is trained to identify defects by learning the distribution of ''normal" tiles.
For each input image
(resized to
pixels), the encoder
maps the image to a
dimensional latent space, while the decoder
attempts to reconstruct the original
pixels. The training objective to maximize the Evidence Lower Bound (ELBO), defined as:
Assume the prior
is a standard normal distribution
.
During a testing phase, the engineers employ a
-VAE variant with
. For a suspicious tile,
the reconstruction mismatch is
and the
contribution is
. Compute the
-VAE
objective value
for this sample.
Published solution
**1. State the beta-VAE objective:** Maximization uses \(\mathcal L_\beta=E[\log p(x\mid z)]-\beta KL\).
**2. Convert mismatch to log likelihood:** A reconstruction mismatch 194.8 contributes -194.8. The specified beta is 4 and this suspicious tile’s KL is separately given as 3.6.
**3. Compute:** \[\mathcal L_\beta=-194.8-4(3.6)=-194.8-14.4=-209.2.\] These suspicious-tile values are not the earlier normal-tile encoder values.
**Answer: -209.2.**
A factory uses a Variational Autoencoder (VAE) to automate the inspection of high-precision
ceramic tiles. The model is trained to identify defects by learning the distribution of ''normal" tiles.
For each input image
(resized to
pixels), the encoder
maps the image to a
dimensional latent space, while the decoder
attempts to reconstruct the original
pixels. The training objective to maximize the Evidence Lower Bound (ELBO), defined as:
Assume the prior
is a standard normal distribution
.
The factory deployment uses an anomaly score
defined as:
If
, the tile is flagged. Determine the anomaly score for the suspicious tile
(reconstruction mismatch:
,
) and state whether it is flagged or not.
6406536642389.
Flagged
6406536642390.
Not flagged
Published solution
**1. Read the anomaly formula:** The deployed score is mismatch plus 0.5 times KL; it is not the beta-VAE objective from the preceding question.
**2. Substitute the suspicious-tile values:** \[A(x)=194.8+0.5(3.6)=194.8+1.8=196.6.\]
**3. Apply the threshold:** Because 196.6>196.0, the tile is flagged. The comparison is strictly greater than, but equality does not arise here.
**Answer: A — Selected: Flagged.**
A: **Correct:** 196.6 exceeds the stated flagging threshold.
B: **Incorrect:** The calculated score is above rather than below the threshold.
. What is the total size (in bits) of the discrete latent
representation (the indices sent to the decoder) for a single input?
6406536642391.
1024 bits
6406536642392.
2048 bits
6406536642393.
4096 bits
6406536642394.
8192 bits
Published solution
**1. Count quantized spatial sites:** The encoder output is 16-by-16-by-32. Each of the \(16(16)=256\) spatial positions selects one codebook vector.
**2. Encode one index:** K=256 possible codes require \(\log_2 256=8\) bits per fixed-length index.
**3. Multiply:** \[256(8)=2048\text{ bits}.\] The 32 floating-point vector coordinates belong to the shared codebook and are not transmitted separately for every site in the requested index representation.
**Answer: B — Selected: 2048 bits.**
A: **Incorrect:** 1024 bits would allocate only four bits per site.
B: **Correct:** 256 sites times eight bits per code index equals 2048.
C: **Incorrect:** 4096 doubles the needed fixed-length index storage.
D: **Incorrect:** 8192 incorrectly multiplies by the vector dimension instead of bits per index.
In a PyTorch DDPM sampling loop, you have a tensor
of shape
representing the
noisy image at step 't'. The model
returns a predicted noise tensor of the same shape.
The timestep 't' is a single integer. When passing 't' to the model, it is typically converted to a
tensor. What must be the shape of this timestep tensor 't' so that it can be correctly processed by
the model for a single image?
6406536642406.
'[1]'
6406536642407.
'[1, 1]'
6406536642408.
'[1, 64]'
6406536642409.
'[64, 64]'
Published solution
**1. Identify batch size:** The noisy image tensor [1,3,64,64] contains one image. Standard DDPM networks take one timestep index per batch item.
**2. Match timestep shape:** The timestep tensor therefore has shape [batch_size]=[1], normally with integer dtype on the appropriate device.
**3. Distinguish embedding:** The model’s timestep-embedding layer later expands that scalar index into a feature vector. The raw timestep input does not need an image-shaped grid or a 64-wide vector.
**Answer: A — Selected: '[1]'.**
A: **Correct:** A one-dimensional tensor of length one supplies one index for the one-image batch.
B: **Incorrect:** An extra singleton axis is not the usual timestep-vector interface.
C: **Incorrect:** The model creates timestep features internally; this is not the raw index shape.
D: **Incorrect:** A spatial image-sized matrix is not needed for one timestep index.
images, the input is first flattened
and then passed through a linear layer 'nn.Linear(3072, 400)'. This is followed by a ReLU activation
and then two parallel linear layers to produce
and
, both mapping from 400 features to a
latent dimension of 20. What is the total number of trainable weight and bias parameters in the
layer that produces the mean vector
?
Published solution
**1. Identify the requested layer:** The mean head is a linear map from 400 hidden features to 20 latent coordinates. The question does not ask for the entire encoder or the parallel variance head.
**2. Count weights:** \(400(20)=8000\).
**3. Count biases and add:** One bias per output gives 20 more, for \[8000+20=8020\] trainable parameters.
**Answer: 8020.**
During the inference phase of a DDPM, a trained U-Net
is used to predict the noise added
to a sample. Once the noise is predicted, the original data point can be estimated using the
formula:
At timestep
, given
and a noisy latent
, the U-Net predicts the noise
. Calculate the estimated clean data point
.
Published solution
**1. Rearrange the forward noising relation:** \[\hat x_0=\frac{x_t-\sqrt{1-\bar\alpha_t}\epsilon_\theta(x_t,t)}{\sqrt{\bar\alpha_t}}.\]
**2. Substitute:** At t=500, \(\bar\alpha=0.7\), xt=0.5 and predicted noise=-0.2, the numerator is \(0.5-\sqrt{0.3}(-0.2)=0.5+0.1095445=0.6095445\).
**3. Divide:** \(\hat x_0=0.6095445/\sqrt{0.7}=0.72856\), approximately 0.73. Subtracting a negative noise prediction increases the numerator.
**Answer: 0.73.**
In a trained Denoising Diffusion Probabilistic Model (DDPM), which of the following statements
correctly describes the inference (sampling) process to generate a new data point
?
6406536642398.
A new sample is generated in a single forward pass by feeding a latent vector
into the trained U-Net.
6406536642399.
To obtain a sample
, the model must iteratively traverse the reverse
decoding process
, starting from Gaussian noise
.
6406536642400.
The inference process is faster than training because it does not require any
forward passes through the neural network.
6406536642401.
During inference, the U-Net predicts the original data point
directly from
without any intermediate steps.
Published solution
**1. Start from noise:** Standard DDPM generation begins with a Gaussian sample xT.
**2. Reverse the corruption gradually:** At each reverse step, a neural network predicts the required noise/denoising quantity and the sampler computes the next less-noisy state. Repeating this follows \(x_T\to x_{T-1}\to\cdots\to x_0\).
**3. Distinguish the model from accelerated variants:** Ordinary DDPM sampling requires repeated network evaluations. A single estimate of x0 can appear inside a reverse update, but that is not the entire standard sampling trajectory.
**Answer: B.**
A: **Incorrect:** A single ordinary U-Net pass does not complete standard DDPM sampling.
B: **Correct:** The standard sampler iteratively denoises from Gaussian noise.
C: **Incorrect:** Sampling repeatedly runs the network; it is not a no-forward-pass procedure.
D: **Incorrect:** Predicting a clean estimate at one timestep does not remove the standard intermediate reverse steps.
**1. Trace coefficient indexing:** With a batch of timestep indices t of shape [B], indexing sqrt_alpha_cumprod[t] returns B coefficients, shape [B].
**2. Match image dimensions:** For images [B,C,H,W], reshape those coefficients to [B,1,1,1]. Then each image receives its own scalar through broadcasting across channels and spatial coordinates.
**3. Choose the valid efficient expression:** sqrt_alpha_cumprod[t][:, None, None, None] adds singleton axes without materializing a full image-sized coefficient tensor. The indexing operation gathers the selected coefficients; the added axes themselves are views.
**Answer: C.**
A: **Incorrect:** gather can select coefficients but leaves shape [B], so it does not by itself align them with the batch axis of [B,C,H,W].
B: **Incorrect:** A view containing a zero-sized dimension cannot reshape a nonempty coefficient vector to the needed shape.
C: **Correct:** It produces [B,1,1,1], which broadcasts correctly and avoids explicit replication.
D: **Incorrect:** Embedding expects a suitable two-dimensional weight table; the given coefficient table is one-dimensional.
