**1. Substitute the density ratio:** The stated divergence is \(\int p_\theta(x)f(p_r(x)/p_\theta(x))dx\).
**2. Use negative logarithm:** For \(f(u)=-\log u\), this becomes \[\int p_\theta(x)\log\frac{p_\theta(x)}{p_r(x)}dx=D_{KL}(p_\theta\|p_r).\]
**3. Check admissibility:** Negative log is convex on positive u and f(1)=0. This is reverse KL in the order requested.
**Answer: B.**
A: **Incorrect:** u log u instead gives forward KL in the stated form.
B: **Correct:** Negative log reverses the density ratio inside the logarithm.
C: **Incorrect:** Half the absolute deviation gives total variation.
D: **Incorrect:** The squared deviation gives a chi-squared-type divergence.
**1. Write the target distance:** Total variation is \(\tfrac12\int|p_r-p_\theta|dx\).
**2. Factor out the reference density:** \[p_\theta\frac12\left|\frac{p_r}{p_\theta}-1\right|=\frac12|p_r-p_\theta|.\]
**3. Match f:** Therefore \(f(u)=|u-1|/2\). It is convex and vanishes at one. The equality uses the usual extended-density convention where needed.
**Answer: A.**
A: **Correct:** This produces half the absolute density difference.
B: **Incorrect:** u log u produces forward KL.
C: **Incorrect:** log(1+u) is not the required normalized convex generator.
D: **Incorrect:** u-1 integrates to zero for normalized densities, rather than total variation.
**1. Set up the conjugate:** \(f^*(t)=\sup_{u>0}[tu-u\log u]\).
**2. Optimize over u:** The derivative is \(t-\log u-1\), giving \(u=e^{t-1}\). The second derivative is -1/u, so this is the maximum.
**3. Substitute:** At the optimizer, \(tu-u\log u=u[t-(t-1)]=u=e^{t-1}\). Thus \(f^*(t)=e^{t-1}\).
**Answer: B.**
A: **Incorrect:** log(1+e^t) is a different conjugate expression.
B: **Correct:** The supremum evaluates to e raised to t-1.
C: **Incorrect:** A quadratic is not the conjugate of u log u on positive u.
D: **Incorrect:** The sigmoid is not the supremum value.
**1. Use the optimal discriminator:** With equal real/fake sampling priors, \(D^*(x)=p_r(x)/(p_r(x)+p_\theta(x))\).
**2. Invert the sigmoid:** The logit is \(a=\log[D/(1-D)]\).
**3. Simplify the ratio:** \[a^*(x)=\log\frac{p_r(x)}{p_\theta(x)}.\] The expression is finite where both densities are positive and extends to infinite logits on one-sided supports.
**Answer: B.**
A: **Incorrect:** This reverses the real/fake density ratio.
B: **Correct:** The log-odds of the optimal real probability equal the log density ratio.
C: **Incorrect:** A difference of densities is not a sigmoid logit.
D: **Incorrect:** Applying a sigmoid again does not invert the original sigmoid.
**1. Substitute D* into the GAN value:** \[V(G,D^*)=\int p_r\log\frac{p_r}{p_r+p_g}+\int p_g\log\frac{p_g}{p_r+p_g}.\]
**2. Introduce the mixture:** With \(m=(p_r+p_g)/2\), each denominator is 2m. The logarithms split into a density-to-m ratio minus log 2.
**3. Add terms:** Both densities integrate to one, producing \[V(G,D^*)=-\log4+D_{KL}(p_r\|m)+D_{KL}(p_g\|m)=-\log4+2JSD(p_r\|p_g).\] The generator minimizes the divergence component.
**Answer: A.**
A: **Correct:** This is the direct optimal-discriminator substitution identity.
B: **Incorrect:** Both the constant sign and divergence sign are reversed.
C: **Incorrect:** The expression involves symmetric mixture KL terms, not only forward KL.
D: **Incorrect:** At matching distributions it equals -log 4, not zero.
. Compare generator gradients w.r.t. the discriminator logit
for:
If
, which magnitude is correct?
6406536376178.
,
6406536376179.
,
6406536376180.
Both are approximately
6406536376181.
Both are approximately
Published solution
**1. Differentiate the sigmoid:** Let D=sigmoid(a), with derivative D(1-D).
**2. Differentiate both losses:** For the saturating minimax loss, \(d\log(1-D)/da=-D\). For the non-saturating loss, \(d[-\log D]/da=-(1-D)\).
**3. Substitute D=0.01:** The magnitudes are 0.01 and 0.99. The non-saturating form supplies a much larger logit-level learning signal when generated samples are confidently rejected; full parameter gradients also involve the network chain rule.
**Answer: B.**
A: **Incorrect:** This reverses the two gradient magnitudes.
B: **Correct:** The derivatives have magnitudes D and 1-D, respectively.
C: **Incorrect:** They are not near one half when D=0.01.
D: **Incorrect:** Only the saturating magnitude is small; the other is near one.
, what is the penalty contribution
for that sample?
6406536376198.
0.3
6406536376199.
9.0
6406536376200.
3.0
6406536376201.
0.9
Published solution
**1. Read the penalty:** For one sampled interpolated input, WGAN-GP contributes \(\lambda(\|\nabla f(\hat x)\|_2-1)^2\).
**2. Substitute:** With gradient norm 1.3 and lambda=10, the deviation is 0.3, whose square is 0.09.
**3. Multiply:** \(10(0.09)=0.9\). The sample expectation is replaced by this sample’s contribution.
**Answer: D — Selected: 0.9.**
A: **Incorrect:** 0.3 is only the unsquared norm deviation.
B: **Incorrect:** 9.0 is ten times the correct penalty.
C: **Incorrect:** 3.0 multiplies the deviation before squaring is accounted for.
D: **Correct:** 10 times (1.3-1) squared equals 0.9.
and the generator matches it perfectly, then the marginal of
generated samples satisfies:
6406536376222.6406536376223.
for a fixed
6406536376224.6406536376225.
Published solution
**1. Match conditional distributions:** Perfect conditional modeling means \(p_\theta(x\mid y)=p_r(x\mid y)\).
**2. Marginalize conditions:** If generation samples conditions from the same data marginal \(p_r(y)\), then \[p_\theta(x)=\int p_\theta(x\mid y)p_r(y)dy=\int p_r(x\mid y)p_r(y)dy=p_r(x).\]
**3. State the required assumption:** The full marginal matches under matched condition sampling. A fixed y produces only its conditional slice; choosing a different label marginal can change the overall generated marginal even with perfect conditional matching.
**Answer: A.**
A: **Correct:** The generated marginal equals the data marginal when conditions are sampled with the matching distribution.
B: **Incorrect:** A fixed conditional slice is not generally the unconditional marginal.
C: **Incorrect:** The distributions over y and x live in different spaces and are not identified this way.
D: **Incorrect:** Equating the generator marginal to an unrelated latent prior is not the conditional marginalization identity.
**1. Optimize pointwise:** With G fixed, the discriminator maximizes \(p_r(x)\log D+p_g(x)\log(1-D)\) at each x.
**2. Set its derivative to zero:** \(p_r/D-p_g/(1-D)=0\), giving \(p_r(1-D)=p_gD\).
**3. Solve:** \[D^*(x)=\frac{p_r(x)}{p_r(x)+p_g(x)}.\] The second derivative is nonpositive, confirming a maximum where the total density is positive. Where both densities are zero, the value is unconstrained.
**Answer: B.**
A: **Incorrect:** This gives the fake probability rather than the real probability.
B: **Correct:** The real density divided by total density is the optimal real/fake posterior.
C: **Incorrect:** A hard threshold discards the calibrated maximizing probability.
D: **Incorrect:** The log ratio is the optimal logit, not the discriminator probability.
**1. Use the idealized support separation:** The optimal real/fake discriminator is zero on generated-only support and one on real-only support.
**2. Examine the minimax loss:** On the generated region where D* is locally constant at zero, \(\log(1-D^*(G(z)))=\log1=0\). Its local input derivative is zero.
**3. Apply the chain rule:** Composing that zero local derivative with the generator derivative gives a vanishing generator gradient in this idealized saturated regime. This conclusion assumes the stated local support separation; it describes the classic saturating-GAN failure mode.
**Answer: B.**
A: **Incorrect:** High curvature does not create a derivative when the composed discriminator is locally constant.
B: **Correct:** Zero discriminator value and local constancy yield the vanishing gradient in the stated idealization.
C: **Incorrect:** This is not generally the gradient of a finite forward KL.
D: **Incorrect:** The loss is locally zero, rather than divergent, at D*=0.
(Classifier-guided sampler) Suppose you want an acceptance probability of the form
and you approximate
If
and
, what is
?
6406536376182.
0.30
6406536376183.
0.60
6406536376184.
0.75
6406536376185.
1.00
Published solution
**1. Recover a density-ratio estimate:** Optimal equal-prior discriminator odds estimate \(p_r/p_g\). With D=0.6, the ratio estimate is \(0.6/(1-0.6)=1.5\).
**2. Divide by the rejection envelope:** M=2 gives \(1.5/2=0.75\).
**3. Apply the cap:** \[\alpha(x)=\min(1,0.75)=0.75.\] The envelope M is assumed to be a valid bound for the intended rejection sampler.
**Answer: C — Selected: 0.75.**
A: **Incorrect:** 0.30 divides the discriminator probability by M without converting to odds.
B: **Incorrect:** 0.60 is D itself rather than the acceptance calculation.
C: **Correct:** The odds 1.5 divided by two give 0.75.
D: **Incorrect:** The cap is inactive because 0.75 is below one.
**1. Condition both participants:** A conditional generator produces x given y, and the discriminator assesses a pair (x,y).
**2. Supply matching real/fake pairs:** Real pairs come from the data joint \(p_r(x,y)\). Fake pairs are made by drawing a condition and a generated x from \(p_\theta(x\mid y)\).
**3. State the adversarial objective:** \[\min_\theta\max_w\left\{E_{(x,y)\sim p_r}\log D_w(x,y)+E_yE_{x\sim p_\theta(\cdot\mid y)}\log[1-D_w(x,y)]\right\}.\] This retains both adversarial terms and the conditioning information.
**Answer: D.**
A: **Incorrect:** The displayed expression omits the conditioning relationship in the discriminator/generator.
B: **Incorrect:** Training only a discriminator on real data is not a conditional GAN game.
C: **Incorrect:** A reconstruction objective alone is not the adversarial conditional GAN objective.
D: **Correct:** Both real and generated pairs are conditioned and included in the minimax game.
In Wasserstein GAN, the discriminator is replaced by a critic
. Which statement is correct?
6406536376190.
due to sigmoid, and must be calibrated probabilities.
6406536376191.
must output the exact density
.
6406536376192.
(no sigmoid) and is constrained to be 1-Lipschitz.
6406536376193.
must be a classifier with softmax outputs.
Published solution
**1. Distinguish critic from classifier:** A Wasserstein critic outputs a real-valued score, not a probability that the input is real.
**2. Enforce the dual constraint:** The Wasserstein-1 dual optimizes over 1-Lipschitz functions, meaning \(|f(x)-f(y)|\le\|x-y\|\).
**3. Apply to the network:** No final sigmoid/softmax is required. Practical training approximates the Lipschitz constraint through methods such as gradient penalties or other suitable constraints.
**Answer: C.**
A: **Incorrect:** A WGAN score is not required to be a sigmoid probability.
B: **Incorrect:** The critic is not a density estimator.
C: **Correct:** A real-valued 1-Lipschitz critic matches the Wasserstein dual.
D: **Incorrect:** A class-probability softmax is not the required output.
**1. State Kantorovich–Rubinstein duality:** For probability distributions with suitable finite first moments,
\[W_1(p_r,p_g)=\sup_{\|f\|_{Lip}\le1}\left(E_{p_r}[f(X)]-E_{p_g}[f(X)]\right).\]
**2. Explain the constraint:** Without a Lipschitz bound, scaling f could increase the objective arbitrarily. The constraint ties score differences to transport distance.
**3. Distinguish alternatives:** Squared transport cost corresponds to a Wasserstein-2-squared objective, while KL and logarithmic discriminator games are different divergences.
**Answer: B.**
A: **Incorrect:** Squared Euclidean transport cost is not the stated W1 dual.
B: **Correct:** This is the difference of expectations over 1-Lipschitz test functions.
C: **Incorrect:** KL is not generally equal to Wasserstein distance.
D: **Incorrect:** Logarithmic classifier scores without the required Lipschitz formulation do not give this dual.
“Latent regression” inversion trains an encoder / regressor
so that:
6406536376206.
and
6406536376207.
and
6406536376208.6406536376209.
directly estimates
Published solution
**1. Identify the inverse direction:** A generator maps latent z to image x. A regressor/encoder for inversion must map image x back to an estimated latent code.
**2. Train on synthetic pairs:** Sample z, generate x=G(z), and learn R(x) to recover z, typically minimizing a latent reconstruction loss.
**3. State the intended relationships:** \(R(x)\approx z\) and \(G(R(x))\approx x\) on the relevant generator range. Exact inversion is not guaranteed when G is non-injective or a target image is outside that range.
**Answer: B.**
A: **Incorrect:** It reverses the generator/encoder input-output spaces.
B: **Correct:** The encoder recovers a latent whose generated image reconstructs the input.
C: **Incorrect:** Equating a discriminator score to an encoder output does not define latent regression.
D: **Incorrect:** The inversion encoder is not directly estimating a normalized image density.
In BiGAN, what does the discriminator receive as “real” vs “fake” inputs?
6406536376210.
Real:
; Fake:
only
6406536376211.
Real:
; Fake:
only
6406536376212.
Real: pairs
with
; Fake: pairs
with
6406536376213.
Real: pairs
sampled jointly from
; Fake: none
Published solution
**1. Construct the data-side joint:** Draw real x from the data distribution and pair it with encoder output E(x).
**2. Construct the generator-side joint:** Draw latent z from its prior and pair it with generated image G(z).
**3. Compare paired distributions:** BiGAN’s discriminator distinguishes \((x,E(x))\) from \((G(z),z)\). Matching these joints trains inference and generation together, rather than discriminating only image marginals or only latent marginals.
**Answer: C.**
A: **Incorrect:** Image-only discrimination describes the basic GAN comparison, not BiGAN’s paired input.
B: **Incorrect:** Latent-only comparison omits the associated image.
C: **Correct:** These are precisely the data-encoder and prior-generator pairs.
D: **Incorrect:** Independent real images and latent samples do not reproduce the data-encoder joint.
In WGAN, why are multiple critic updates typically performed per generator update? The most
accurate reason is:
6406536376214.
The generator has no gradients unless the critic is exactly optimal, so you
must fully converge the critic each time.
6406536376215.
The critic should be kept closer to its optimum so its gradients provide a
meaningful approximation to the Wasserstein distance and yield informative directions for
updating θ.
6406536376216.
Multiple critic steps ensure the critic output becomes a probability.
6406536376217.
It is required only when using sigmoid outputs.
Published solution
**1. Recall the ideal objective:** The optimized Lipschitz critic approximates a test function that witnesses the Wasserstein distance between current real and generated distributions.
**2. Explain extra critic steps:** Several critic updates can keep its scores and input gradients closer to a useful solution for the current generator, improving the quality of generator-update directions.
**3. Avoid an excessive claim:** The critic need not be solved exactly for every nonzero gradient to exist. More steps do not convert its real-valued output into a probability.
**Answer: B — Selected: The critic should be kept closer to its optimum so its gradients provide a meaningful approximation to the Wasserstein distance and yield informative directions for updating θ.**
A: **Incorrect:** Nonzero generator gradients do not require an exactly optimal critic.
B: **Correct:** A better-adapted critic can provide more informative approximate Wasserstein gradients.
C: **Incorrect:** The critic remains a score function rather than a calibrated probability.
D: **Incorrect:** Standard WGAN does not rely on sigmoid critic outputs.
**1. Use the one-dimensional Gaussian feature formula:** \[FID=(\mu_r-\mu_g)^2+(\sigma_r-\sigma_g)^2.\]
**2. Substitute:** \(\mu_r=0\), \(\mu_g=1\), \(\sigma_r=2\), \(\sigma_g=1\). The mean contribution is \((0-1)^2=1\) and the standard-deviation contribution is \((2-1)^2=1\).
**3. Add:** FID=2. The given sigma quantities are standard deviations, not variances to be substituted without taking their roots.
**Answer: B — Selected: 2.**
A: **Incorrect:** One is only one of the two mismatch contributions.
B: **Correct:** The mean and spread contributions each equal one, totaling two.
C: **Incorrect:** Three is not the sum of the stated squared differences.
D: **Incorrect:** Five would confuse the covariance interaction or standard-deviation terms.
At an ideal BiGAN equilibrium (infinite capacity, optimal discriminator), the matched joint
distributions imply most directly that:
6406536376226.
is independent of
.
6406536376227.
The joint distributions of
and
match.
6406536376228.
Only marginals match:
, but nothing about
.
6406536376229.
must be linear.
Published solution
**1. Identify the distributions being matched:** BiGAN compares the joint law induced by drawing real x and encoding it, with the joint law induced by drawing latent z and generating x.
**2. State ideal equilibrium:** With unlimited capacity and successful optimization, \[(X,E(X))\overset d=(G(Z),Z).\]
**3. Explain why this is stronger:** Joint matching includes compatible relationships between samples and latent codes, not just separate marginals. It neither requires a linear generator nor makes its encoding independent of the input.
**Answer: B.**
A: **Incorrect:** The encoding is coupled with x rather than required to be independent.
B: **Correct:** Matching the data-encoder and generator-latent joints is the direct equilibrium statement.
C: **Incorrect:** Joint matching imposes more than marginal image matching alone.
D: **Incorrect:** No linearity requirement follows.
**1. Identify the discriminator’s goal:** It maximizes log probability of correctly marking real data as real and generated data as fake.
**2. Identify the generator’s minimax goal:** The generator minimizes that same discriminator value, trying to make generated data indistinguishable.
**3. State the original game:** \[\min_\theta\max_w\left[E_{x\sim p_r}\log D_w(x)+E_{z\sim p_z}\log(1-D_w(G_\theta(z)))\right].\] The non-saturating generator alternative is useful in practice but is not the original shared minimax expression.
**Answer: A.**
A: **Correct:** This has the correct minimization/maximization order and both real/fake terms.
B: **Incorrect:** The players’ optimization directions are reversed.
C: **Incorrect:** The generated-data term and adversarial structure are missing.
D: **Incorrect:** Direct negative log-likelihood fitting is not the original GAN minimax objective.
**1. Hold the generator fixed:** The trained mapping G is available, and the target image x is given. Optimization inversion searches for an input code, not new discriminator weights.
**2. Minimize reconstruction error:** \[z^*\in\arg\min_z\|x-G(z)\|_2^2.\] One can backpropagate through fixed G to update z.
**3. Interpret the result:** G(z*) approximates x within the generator’s image range. The minimizing code need not be unique, and exact reconstruction is not guaranteed.
**Answer: A.**
A: **Correct:** This searches the latent code for the fixed target image.
B: **Incorrect:** Discriminator optimization is not latent inversion.
C: **Incorrect:** Changing generator parameters for a random latent is a different fitting task.
D: **Incorrect:** Optimizing the target image rather than the latent does not solve the stated inverse problem.
