Deep Learning Practice · Quiz 2 · Jan 2026

**1. Understand the model output:** Wav2Vec 2.0 produces one hidden vector per audio time step. Its last hidden state has shape \((B,T,H)\), where \(B\) is batch size, \(T\) is sequence length and \(H\) is hidden size. Longer recordings generally produce larger \(T\). **2. Pool across time:** The code uses mean(dim=1). For each hidden feature it averages the values over time: \[z_{b,h}=\frac{1}{T}\sum_{t=1}^{T}h_{b,t,h}.\] The result has shape \((B,H)\), regardless of the original recording length. **3. Use the summary:** A language classifier can now accept one fixed-length vector per recording. For padded batches, a padding-aware mean should ignore padded time steps. **Answer: B — Selected: To convert a variable-length sequence of hidden vectors into a single fixed- length representation that summarizes the entire utterance.** A: **Incorrect:** Pooling averages hidden features; it does not resample the waveform. B: **Correct:** Averaging the time dimension gives one fixed-size utterance representation. C: **Incorrect:** A mean includes all valid time steps, rather than selecting only amplitude peaks. D: **Incorrect:** Pooling produces an embedding, not a reconstructed waveform.

**1. Identify unnecessary work:** Feature extraction needs a forward pass, but does not need a backward graph for updating the speech model. **2. Disable gradient recording:** Run the forward pass inside with torch.no_grad():. Intermediate tensors are then not saved for backpropagation, reducing memory use. Requesting output_hidden_states=True controls which representations are returned; it does not disable gradients. **3. Separate inference mode from gradient mode:** model.eval() is also normally used to disable training behavior such as dropout. It serves a different purpose from no_grad(). Weights change only if an update is performed; no_grad() itself is not an optimizer. **Answer: B.** A: **Incorrect:** model.train() enables training behavior and does not turn off autograd. B: **Correct:** The context manager prevents recording the backward graph for the extraction block. C: **Incorrect:** requires_grad belongs to parameter tensors; repeatedly altering every layer is unnecessary and is not the shown API. D: **Incorrect:** squeeze changes tensor dimensions; it does not disable gradient recording.

**1. Recognize two input types:** Whisper uses acoustic features from audio and token IDs from the reference transcript. **2. Combine preprocessing:** WhisperProcessor wraps WhisperFeatureExtractor and WhisperTokenizer. The feature extractor prepares audio features, while the tokenizer converts text into token IDs and decodes predicted IDs into text. **3. Keep roles separate:** This processor organizes preprocessing and padding. The neural model performs prediction, and a separate metric implementation computes WER. Audio must be supplied at the required sampling rate; declaring that rate is not a substitute for resampling. **Answer: B — Selected: It is a wrapper class that combines a WhisperFeatureExtractor (for audio) and a WhisperTokenizer (for text) into a single object to simplify data preprocessing and padding.** A: **Incorrect:** A processor is not an ASR prediction model. B: **Correct:** It bundles the audio feature extractor and text tokenizer. C: **Incorrect:** Actual waveform resampling is handled by the data/audio processing pipeline, not merely by the processor wrapper. D: **Incorrect:** WER compares predicted words with reference words; this is not the processor role.

**1. Separate content and voice:** Text tells a text-to-speech system what to say. Different speakers can say those same words with different vocal characteristics. **2. Condition on the speaker:** A speaker embedding is a numerical vector representing voice identity. SpeechT5 receives it alongside the text representation so that generated speech can reflect the reference speaker. **3. Interpret its role:** It conditions the voice, rather than translating words, denoising recordings or directly controlling inference speed. **Answer: B — Selected: To define the specific vocal characteristics (voice identity) of the speaker.** A: **Incorrect:** Translation changes language content, rather than specifying voice identity. B: **Correct:** The vector conditions the generated speech on speaker characteristics. C: **Incorrect:** Denoising is a separate signal-processing task. D: **Incorrect:** An embedding is a conditioning input, not a speed optimization.

**1. Given:** The waveform contains \(N=13{,}440{,}000\) samples at a sampling rate \(f_s=48{,}000\) samples per second. **2. Use the duration formula:** Each second contains \(f_s\) samples, so \[t=\frac{N}{f_s}=\frac{13{,}440{,}000}{48{,}000}=280\text{ seconds}.\] **3. Check:** \(280\times48{,}000=13{,}440{,}000\). The recording is 4 minutes 40 seconds long. **Answer: 280.**

**1. Write the reference and prediction:** Reference: “the quick brown fox jumps over a lazy dog”. Prediction: “a quick hungry fox jumps over lazy dog”. The reference has \(N=9\) words. **2. Align the words:** “the” becomes “a” and “brown” becomes “hungry”, giving two substitutions. The article “a” before “lazy” is missing, giving one deletion. All other words match, and there are no insertions. **3. Compute WER:** With substitutions \(S=2\), deletions \(D=1\), insertions \(I=0\), \[\mathrm{WER}=\frac{S+D+I}{N}=\frac{2+1+0}{9}=\frac13\approx0.3333.\] This is approximately 33.33%; the numeric field expects the fraction. **Answer: 0.3333.**

**1. Define diarization:** Speaker diarization answers “who spoke when?” It groups speech segments by voice identity without requiring the actual names of speakers. **2. Follow the identity pipeline:** A speaker encoder turns each speech segment into an embedding. A similarity or distance measure compares embeddings. A clustering algorithm groups similar embeddings into speaker clusters. **3. Separate voice from text:** Grammar and named entities describe a transcript, but do not by themselves establish that two recordings contain the same voice. **Answer: A, C, E — Selected: Speaker Embedding Extraction: Generating numerical representations (e.g., x- vectors) that capture the identity-bearing features of the vocal signal.; Clustering: Applying algorithms like Spectral Clustering or AHC to group embeddings into clusters corresponding to individual speakers.; Distance/Similarity Metric Computation: Calculating the mathematical relationship (e.g., Cosine similarity or distances) between different speech segments.** A: **Correct:** Speaker embeddings retain information useful for distinguishing voices. B: **Incorrect:** POS tagging identifies grammatical categories, not acoustic identity. C: **Correct:** Clustering groups similar speaker embeddings into individual speaker groups. D: **Incorrect:** A named entity in the transcript need not be the person currently speaking. E: **Correct:** Distances or similarities provide the comparison needed to group segments.

**1. Identify Whisper’s core function:** Whisper is an automatic speech recognition model. It turns speech into a transcript. **2. Connect transcript and speaker segments:** A diarization system can assign speaker labels and use Whisper transcription as its text component. Word timestamps, obtained through a supported Whisper decoding/alignment implementation, help associate individual words with speaker intervals. **3. Keep identity processing separate:** A speaker embedding model and a clustering method are still needed; standard Whisper is not a dedicated replacement for either component. **Answer: A, B — Selected: As the automatic speech recognition backend for transcription after diarization; To provide word-level timestamps for aligning speaker segments.** A: **Correct:** Its transcript can be combined with diarization output. B: **Correct:** Word timestamps support alignment of transcript words with speaker segments. C: **Incorrect:** Whisper is not a speaker-embedding clustering algorithm. D: **Incorrect:** Its standard ASR interface is not a dedicated speaker-identity encoder.

**1. Prepare compatible audio:** Feed speech waveforms at the sample rate, channel layout and amplitude normalization expected by the speaker model. Use its prescribed preprocessing, rather than applying arbitrary normalization twice. **2. Use hardware efficiently:** Move model and inputs to the same GPU when available. Batch compatible speech segments to improve throughput, with padding/length information as required. **3. Use inference settings:** Apply evaluation mode and torch.no_grad() for embedding extraction. The input is audio; text tokens do not contain the vocal information needed by a speech encoder. **Answer: A, B, C, E — Selected: Use .to(device) to move models to GPU; Normalize audio waveforms before feeding to the embedding model; Use batch processing for extracting embeddings from multiple segments; Use torch.no_grad() during inference to save memory.** A: **Correct:** Moving the model to the available GPU enables GPU execution; inputs must follow it. B: **Correct:** Consistent model-appropriate normalization helps match training-time audio conditions. C: **Correct:** Batching can improve throughput; account for padding and memory limits. D: **Incorrect:** A speaker encoder expects waveform/features, not raw text tokens. E: **Correct:** Disabling gradient recording avoids storing an unnecessary backward graph.

**1. Read blank A:** The code applies a method to the dataset with arguments "audio" and Audio(sampling_rate=16000). It is changing how the audio column is represented and decoded. **2. Use the dataset API:** The intended expression is dataset.cast_column("audio", Audio(sampling_rate=16000)). This specifies an audio feature that decodes/resamples recordings to the required rate when accessed. **3. Explain why it matters:** Whisper’s feature extractor expects correctly sampled audio. Merely changing a sampling-rate number without resampling the samples would change their interpretation incorrectly. **Answer: B — Selected: cast_column.** A: **Incorrect:** apply is not the shown Hugging Face Dataset column-casting method. B: **Correct:** cast_column changes the audio column feature to the requested Audio type. C: **Incorrect:** transform is not the required method for this expression. D: **Incorrect:** batch_decode converts predicted token IDs to text, not waveform sampling rates.

**1. Read the surrounding code:** The variable tokenizer is initialized using from_pretrained(model_id, language="Assamese", task="transcribe"). Later it processes batch["sentence"] and returns input_ids. **2. Choose the text component:** WhisperTokenizer maps Assamese reference text into token IDs and sets the relevant language/task token behavior. **3. Distinguish other components:** The script already creates a separate audio feature extractor. Here it needs the dedicated tokenizer class, rather than the model, trainer arguments or combined processor. **Answer: A — Selected: WhisperTokenizer.** A: **Correct:** WhisperTokenizer is the dedicated component used for the sentence-to-token operation. B: **Incorrect:** A processor bundles components, but the shown variable and later text-only use call for the tokenizer. C: **Incorrect:** This class performs neural generation, rather than text tokenization. D: **Incorrect:** Training arguments configure optimization and do not tokenize text.

**1. Trace blank C:** Its input is audio["array"], together with the recording sampling rate. Its output is accessed through .input_features[0]. **2. Match input and output:** WhisperFeatureExtractor converts a waveform into the log-mel acoustic features used by Whisper. The initialized variable is feature_extractor. **3. Keep labels separate:** The tokenizer processes the transcript into labels in the next part of the function. Audio features and text labels have different preprocessing routes. **Answer: A — Selected: feature_extractor.** A: **Correct:** This is the initialized audio-to-feature component, and it returns input_features. B: **Incorrect:** The tokenizer processes text, not waveform samples into acoustic features. C: **Incorrect:** A collator combines/pads examples into a batch; it is not the feature extraction call. D: **Incorrect:** feature_extractor is a matching option.

**1. Read the intended nested structure:** extract_all_chars returns "vocab": [vocab], where vocab itself is a list of characters. Thus each intended entry v in vocabs["vocab"] contains its character list at v[0]. **2. Flatten and remove duplicates:** Start with an empty vocab_list. Extend it using v[0] for each entry, then apply vocab_list = list(set(vocab_list)). The set ensures that every character occurs only once; list converts it back to a list. Ordering is not guaranteed by this expression. **3. Select the matching code:** Only C performs both the intended flattening and final deduplication. **Implementation note:** The printed Dataset output is an assumption in the question. As written, map with batched=True can return fewer rows than the retained text column, causing an Arrow length error. The runnable version should remove the original text column during that batched map. This does not change the character-union operation asked here. **Answer: C.** A: **Incorrect:** It flattens the nested lists but does not remove repeated characters across entries. B: **Incorrect:** It traverses one level differently and does not explicitly deduplicate the final list. C: **Correct:** It extends by v[0] and then uses a set to retain only unique characters. D: **Incorrect:** dict does not compute a character union and can reject the nested list structure.

**1. Combine the source text:** The two strings are “this is first sentence” and “this is second sentence”. Spaces count as characters because the code takes set(all_text), without removing spaces. **2. List unique characters:** The distinct letters are c, d, e, f, h, i, n, o, r, s and t. There are 11 letters. Repeated occurrences of letters are counted only once. **3. Include the space:** Add the single distinct space character: \(11+1=12\). The order of the vocabulary list may vary, but its length remains 12. **Answer: 12.**

**1. Identify the intermediate representation:** A TTS acoustic model often predicts a mel-spectrogram, which describes sound energy over time and frequency. It is not yet the waveform sent to a speaker. **2. Convert features to sound:** A vocoder maps those acoustic features into a sequence of waveform samples. For example, a compatible HiFi-GAN vocoder can turn a predicted spectrogram into audible speech. **3. Locate it in the pipeline:** Text processing comes first, acoustic prediction follows, and waveform synthesis by the vocoder produces playable audio. **Answer: C — Selected: To convert intermediate acoustic features (like mel-spectrograms) into audible waveforms.** A: **Incorrect:** Translation changes the words/language, rather than synthesizing a waveform. B: **Incorrect:** Tokenization divides input text into model tokens. C: **Correct:** The vocoder synthesizes waveform samples from acoustic features. D: **Incorrect:** ZIP compression is a file-storage operation.

**1. Understand sampling rate:** A sample rate specifies the number of audio samples per second. The same array interpreted at the wrong rate represents different timing and frequencies. **2. Match the speaker encoder:** The reference speaker encoder used in this workflow expects 16,000-Hz audio because that matches its training/input specification. Actual resampling makes the waveform compatible with that expectation. **3. Distinguish related operations:** Amplitude normalization and resampling are different operations. The reason for 16 kHz here concerns the selected speaker encoder; it is not a universal limit of Python, human hearing or every TTS component. **Answer: D — Selected: Because the model was specifically trained on 16kHz audio data.** A: **Incorrect:** Increasing file size is not the objective. B: **Incorrect:** Python audio libraries can process many other sample rates. C: **Incorrect:** 16 kHz is a sample rate, not the maximum audible frequency; its Nyquist limit is 8 kHz. D: **Correct:** The selected reference speaker model expects audio matching its 16-kHz training setup.