Deep Learning Practice · Quiz 2 · Sep 2025

**1. Describe the Monte Carlo target:** A sampled return \(G_i\) is the discounted sum of rewards following a visit to a state under the policy. Batch Monte Carlo holds the collected return targets fixed while fitting state values. **2. Minimize squared error:** For visits to one state, the objective is \(\sum_i(v-G_i)^2\). Differentiating gives \(2\sum_i(v-G_i)=0\), hence \(v=\sum_iG_i/n\). **3. Interpret:** In the tabular setting, the least-squares fit is the empirical mean of the appropriate sampled returns. With function approximation, batch fitting analogously minimizes errors to those return targets. Thus the statement is true. **Answer: A — True.** **Convention note:** This general reinforcement-learning question is present in the exported DLP paper. It is retained with its exact original paper/question mapping; the adjacent trajectory-specific question cannot be solved from the supplied pages. A: **Correct:** Batch Monte Carlo fits sampled return targets, yielding their sample means in the tabular least-squares setting. B: **Incorrect:** This rejects the correct least-squares characterization of batch Monte Carlo return fitting.

**1. Separate representation from grouping:** A speaker encoder first maps speech segments to embeddings that summarize voice characteristics. Clustering operates on those already extracted vectors. **2. Explain agglomeration:** Agglomerative clustering starts with small clusters and repeatedly merges the most similar ones according to a distance and linkage rule. A speaker count or distance threshold can stop the process. **3. Connect to diarization:** The resulting clusters serve as speaker identities for time-labeled segments. The algorithm does not transcribe words, create embeddings or detect silence by itself. **Answer: C — To compare the cosine distance between segment embeddings and iteratively .** **Convention note:** Cosine distance is a common choice in the pipeline described, but other distances, linkage rules and threshold-based stopping criteria are possible. A: **Incorrect:** Transcription predicts spoken words and is an ASR task rather than clustering. B: **Incorrect:** The embedding model extracts vectors; agglomeration groups vectors already available. C: **Correct:** Comparing embedding similarity and progressively merging clusters is the intended agglomerative stage. D: **Incorrect:** VAD identifies speech intervals before or alongside embedding extraction rather than grouping speaker vectors.

**1. Separate acoustic modeling from synthesis:** A TTS acoustic model converts text or phoneme representations into a time–frequency representation such as a mel-spectrogram. That representation is not yet an audio waveform. **2. Identify the vocoder:** HiFi-GAN or a suitably conditioned WaveNet generates waveform samples from the acoustic representation, enabling audible playback. **3. Match the named component:** Text normalization, phoneme conversion, duration prediction and speaker encoding occur elsewhere in the pipeline. The named component’s role is waveform synthesis. **Answer: C — To convert the intermediate mel-spectrogram representation into a raw audio .** A: **Incorrect:** Grapheme-to-phoneme conversion is a front-end task rather than the vocoder’s waveform-generation role. B: **Incorrect:** Duration prediction aligns text units with time and is not the vocoder’s central job. C: **Correct:** The neural vocoder turns the mel/acoustic representation into waveform samples. D: **Incorrect:** A speaker encoder extracts voice characteristics; a vocoder synthesizes the audio signal.

**1. Read the standard output axes:** A Wav2Vec2 hidden-state tensor uses batch, time and feature axes. In (1,49,768), one is the batch size. **2. Explain 49:** The convolutional feature encoder downsamples the 16,000 waveform samples into 49 time positions for the stated model. Kernel widths and boundary handling matter in addition to the overall stride. **3. Explain 768:** Each output time position has a learned representation with 768 features. This is a hidden dimension, not a token vocabulary or the number of phonemes. **Answer: C — 49 = the downsampled sequence length (timesteps); 768 = the model’s hidden .** A: **Incorrect:** The first axis, not 768, is batch size; 49 counts time positions. B: **Incorrect:** This exchanges the time and feature dimensions. C: **Correct:** 49 is the downsampled time length and 768 is the representation width at each time step. D: **Incorrect:** Neither 49 nor 768 denotes attention heads or output vocabulary in this hidden-state tensor.

**1. Locate the CTC projection:** The encoder outputs a hidden vector at each time position. A final linear map converts each vector into one score for each output token, including the CTC blank. **2. Identify logits:** These values are raw unnormalized scores. For batch size B, time length T and vocabulary size V, the logits have shape \(B\times T\times V\). Softmax or log-softmax can convert them into normalized values for training or decoding. **3. Separate scores from transcription:** A decoder still needs to select an alignment and apply CTC collapse rules, including merging appropriate repeats and removing blanks, to produce a text string. **Answer: C — Raw, unnormalized scores for each token in the vocabulary (including the .** A: **Incorrect:** The logits precede decoding and are not already a final string. B: **Incorrect:** This CTC head predicts text tokens, not speaker identities. C: **Correct:** The tensor provides a raw vocabulary score at every time position, including blank. D: **Incorrect:** An acoustic time–frequency input representation is different from the token-level output logits.

**1. Identify the autoregressive bottleneck:** Tacotron-style decoding predicts acoustic frames sequentially, using previous outputs and learned attention alignments. This limits parallel generation and can cause alignment errors. **2. Explain the duration-based alternative:** FastSpeech predicts phoneme durations. A length regulator repeats or expands phoneme representations according to those durations to create a frame-length sequence. **3. Explain the benefit:** Acoustic frames can then be generated in parallel rather than through an autoregressive attention loop. Explicit durations also make speaking-rate control straightforward. **Answer: C — It replaces the auto-regressive attention mechanism with a parallel ”Duration .** A: **Incorrect:** The principal speed innovation is in acoustic generation and alignment, not simply choosing a newer vocoder. B: **Incorrect:** Original FastSpeech generates mel-spectrograms rather than replacing them with direct raw waveform output. C: **Correct:** Duration prediction and length regulation enable parallel non-autoregressive acoustic generation. D: **Incorrect:** A larger encoder alone does not explain the removal of sequential acoustic decoding.

**1. Count the original samples:** Five seconds at 44,100 samples per second gives \(5(44{,}100)=220{,}500\) samples per channel. **2. Count the resampled signal:** Preserving five seconds at 22,050 samples per second gives \(5(22{,}050)=110{,}250\) samples per channel. **3. Compare:** \(110{,}250/220{,}500=1/2\), so the count is halved. Proper resampling also low-pass filters to prevent aliasing; simply relabeling the sample rate would instead change playback duration. **Answer: B — The number of samples is halved..** A: **Incorrect:** The duration is fixed while the sample rate decreases, so the sample count must change. B: **Correct:** Halving the sample rate halves the count over the same five-second duration. C: **Incorrect:** The count decreases rather than doubles. D: **Incorrect:** A quarter would require a fourfold reduction in sample rate rather than the stated twofold reduction.

**1. Identify the loading requirement:** The goal is to start iterating over a large dataset without first downloading and materializing the entire collection. This is an on-demand data-access requirement. **2. Choose the Datasets mechanism:** Passing streaming=True to load_dataset returns a streaming iterable dataset. Examples are read as iteration progresses, avoiding the full up-front download in supported dataset formats. **3. Distinguish other resource settings:** Batch size controls grouping of examples, and disk-backed datasets can avoid holding everything in RAM, but neither is the specific switch for streaming access. **Answer: C — streaming = True.** A: **Incorrect:** lazy_load=True is not the documented switch for this load_dataset streaming behavior. B: **Incorrect:** A batch-size setting does not select on-demand dataset streaming. C: **Correct:** streaming=True is the supported load_dataset option for an IterableDataset accessed as it is consumed. D: **Incorrect:** in_memory=False is not the documented streaming switch; disk-backed storage and streaming are distinct concepts.

**1. Find speech intervals:** Voice activity detection excludes silence and identifies material from which speaker characteristics can be estimated. **2. Represent each segment:** A speaker encoder produces an embedding intended to capture voice identity rather than the exact spoken text. Segment times are retained for the final timeline. **3. Group and label:** Clustering groups embeddings attributed to the same speaker, yielding “who spoke when.” Synthesizing new audio with TTS is not required for that output. **Answer: A, B, C — Voice Activity Detection (VAD) to filter out silence.; A speaker embedding model (e.g., ECAPA-TDNN) to create vectors for speech ; A clustering algorithm (e.g., Agglomerative Clustering) to group segments by .** **Convention note:** This is the conventional modular embedding-and-clustering pipeline asked about. End-to-end diarization systems need not expose all three as separate modules. A: **Correct:** VAD identifies usable speech intervals and filters silence in the described modular pipeline. B: **Correct:** A speaker embedding model supplies the voice representations needed for embedding-based grouping. C: **Correct:** Clustering assigns similar segment representations to speaker groups. D: **Incorrect:** TTS generates speech; a diarization output is a timed speaker labeling rather than synthesized audio.

**1. Write the output-length rule:** For a valid 1D convolution, \[L_{out}=\left\lfloor\frac{L_{in}+2p-d(k-1)-1}{s}\right\rfloor+1,\] where p is padding, d dilation, k kernel size and s stride. **2. Examine stride, padding and kernel size:** Increasing s spaces outputs farther apart; decreasing p supplies fewer padded positions; increasing k makes each kernel cover a wider interval. With other parameters fixed, these changes tend to reduce output length. **3. Separate channels from time:** Increasing input channels changes the number of values combined at each time position and the parameter count. It does not appear in the length formula. **Answer: A, B, C — Increase stride, decrease padding, or increase kernel size.** **Convention note:** Because of the floor operation, a particular small parameter change can leave output length unchanged. The selected options are the changes with the stated shortening effect in the usual nondegenerate setting, not a strict decrease for every possible integer configuration. A: **Correct:** A larger stride generally yields fewer output positions for a fixed input and receptive field. B: **Correct:** Less padding reduces the available spatial/time extent and can shorten the output. C: **Correct:** A larger kernel with fixed stride, dilation and padding can reduce output length. D: **Incorrect:** Input channel count does not enter the output-sequence length formula.

**1. Distinguish processor from neural encoder:** Wav2Vec2Processor combines an audio feature extractor/preprocessor with a tokenizer for text labels. The processor handles input/label preparation; the neural model learns the acoustic representations. **2. Explain pretraining:** Wav2Vec2 learns from unlabelled audio by masking latent time positions and distinguishing the appropriate quantized target representation from distractors. It is not pretrained as supervised text-to-speech. **3. Identify the architecture:** The contextual network is a Transformer encoder acting on convolutionally extracted audio features. An ASR fine-tuning head can subsequently map contextual states to CTC token scores. **Answer: A, C, D — A, C and D — processor functions, masked acoustic pretraining and encoder architecture.** A: **Correct:** The processor combines audio preprocessing with the tokenizer used for text labels. B: **Incorrect:** The pretraining objective is self-supervised acoustic representation learning, not supervised TTS. C: **Correct:** Masked latent positions are trained against quantized acoustic targets using a contrastive objective. D: **Correct:** Its contextual Transformer is encoder-only, preceded by a convolutional audio feature extractor.

**1. Identify the shared architecture:** SpeechT5 uses an encoder–decoder Transformer with modality-specific input/output components, supporting speech and text tasks. **2. Explain voice conditioning:** In the TTS setup, a speaker embedding supplies information about the voice to generate. The acoustic output can then be converted into a waveform with a compatible external vocoder. **3. Separate architecture from a particular pipeline:** HiFi-GAN is commonly paired with SpeechT5 but is not a mandatory integrated component of every SpeechT5 architecture. Pretraining uses speech and text rather than text alone. **Answer: A, B — It is an encoder-decoder Transformer model.; To generate speech for a specific voice, it requires a speaker embedding as an .** A: **Correct:** SpeechT5’s shared backbone is an encoder–decoder Transformer. B: **Correct:** The described speaker-conditioned TTS generation supplies a speaker embedding as an additional input. C: **Incorrect:** A compatible vocoder is needed for waveform synthesis, but HiFi-GAN is a pipeline choice rather than an inseparable mandatory backbone component. D: **Incorrect:** SpeechT5 is pretrained using speech and text modalities, not only text.

**1. Compute the accumulation factor:** On one GPU, effective batch size is \(B_{micro}\times A\), where A is the number of accumulated microbatches. With \(B_{micro}=8\) and target 64, \(A=64/8=8\). **2. Explain activation savings:** Gradient checkpointing saves fewer intermediate activations and recomputes them during backpropagation. It can help fit a chosen microbatch at the cost of extra computation. **3. Distinguish direct solution from assistance:** Accumulation eight directly gives effective batch 64 without raising the per-device batch. Checkpointing is a complementary memory aid, not itself a change to effective batch size. **Answer: A, C — A; C is a complementary memory-saving setting.** **Convention note:** C is included under “could help”: checkpointing assists memory usage but does not by itself turn an effective batch of eight into 64. A is the setting that directly satisfies the target. A: **Correct:** Eight microbatches of eight examples supply the target effective batch of 64 on one device. B: **Incorrect:** An extremely small learning rate changes optimization dynamics, not the number of examples per effective batch or available VRAM. C: **Correct:** Checkpointing can reduce activation memory and help the training configuration fit, but still needs accumulation or another batching change to achieve 64. D: **Incorrect:** A per-device batch of 64 violates the stated memory limit of eight examples.

**1. Align the sentences:** The reference is “the quick brown fox jumps over the lazy dog”. The prediction is “the quick fox jumped over a lazy dog”. Delete “brown”; substitute “jumps” with “jumped”; and substitute the second “the” with “a”. **2. Count edit types:** The optimal word alignment has one deletion, two substitutions and zero insertions. The reference contains nine words. **3. Calculate WER:** \[\operatorname{WER}=\frac{S+D+I}{N}=\frac{2+1+0}{9}=\frac13=0.3333\ldots.\] Rounded to two decimals, the answer is 0.33. The denominator is reference length, not hypothesis length. **Answer: 0.33.**