Deep Learning Practice · Quiz 1 · Sep 2025

**1. Separate the toolkit from the merge algorithm:** SentencePiece can implement BPE or unigram tokenization. Its key distinction from a conventional word-pretokenized BPE pipeline is that it learns directly from raw Unicode text. **2. Explain language independence:** It does not require a language-specific word segmenter or whitespace-defined word boundaries before learning subwords. This is useful for scripts where spaces do not reliably separate words. **3. Select the central advantage:** Language-agnostic raw-text processing is the defining benefit described here. Training speed, vocabulary size and deterministic versus stochastic segmentation depend on configuration rather than this universal contrast. **Answer: C — It is inherently language-agnostic, treating text as a raw stream of Unicode .** A: **Incorrect:** SentencePiece’s advantage is not a guarantee of avoiding pair counting or always training faster. B: **Incorrect:** The vocabulary is configured, and the algorithm does not universally merge the shortest tokens first. C: **Correct:** Raw-text Unicode processing avoids a mandatory language-specific word segmentation stage. D: **Incorrect:** SentencePiece supports subword regularization and sampling; deterministic behavior is not its defining contrast with BPE.

**1. Count parameter-sized arrays:** In ordinary full fine-tuning, weights occupy one value per parameter and gradients occupy another. Adam also maintains first- and second-moment states, each with one value per trained parameter. **2. Compare equal-precision storage:** At 32 bits, a 10-billion-parameter array uses 40 GB in decimal units. Weights use 40 GB, gradients 40 GB, and the two Adam states together use 80 GB. **3. Identify the listed category:** The combined optimizer states are therefore the largest of the listed persistent categories under the usual Adam accounting. Activations, master copies and sharding can alter total memory, but are separate implementation considerations. **Answer: C — The optimizer states (e.g., momentum and variance)..** **Convention note:** This compares conventional unsharded Adam storage. Precision, optimizer compression, offloading and activation memory can change which category dominates in a particular implementation. A: **Incorrect:** One weight array is smaller than the two parameter-sized Adam moment arrays at the same precision. B: **Incorrect:** One gradient array has one value per parameter, versus two moment values for Adam states. C: **Correct:** Adam’s first- and second-moment estimates together commonly exceed weight or gradient storage alone. D: **Incorrect:** Embeddings are already part of model parameters; vocabulary metadata is not the dominant additional persistent category.

**1. Interpret the feedback:** Ideal human responses can provide a supervised starting point, while rankings compare alternative model-generated responses according to human preferences. **2. Connect rankings to adaptation:** In the classic RLHF pipeline, preference comparisons train a reward model, and reinforcement learning adjusts the policy toward higher-reward responses. This is designed to improve behavior such as helpfulness and safety. **3. Distinguish the closest alternative:** Instruction tuning uses demonstrations, but the explicit response-ranking and preference-alignment setup points to RLHF as the most complete answer among the choices. **Answer: C — Reinforcement Learning from Human Feedback (RLHF).** A: **Incorrect:** Instruction tuning can use ideal demonstrations, but does not by itself describe the preference-ranking and reinforcement-learning alignment stage. B: **Incorrect:** Zero-shot prompting changes the input instruction without training from the human rankings. C: **Correct:** RLHF explicitly uses human preference feedback to guide policy adaptation. D: **Incorrect:** Continued pretraining learns from additional language-model data rather than directly optimizing human-ranked preferences.

**1. Identify the risky update source:** A very large gradient norm can cause an unusually large parameter update during fine-tuning. Gradient clipping rescales such a gradient while keeping its direction. **2. Apply the clipping rule:** With threshold one, \(g\) is replaced by \(g\min(1,1/\|g\|)\), apart from numerical stabilization. The TrainingArguments setting controlling this is max_grad_norm=1.0. **3. Limit the claim:** This is the option directly targeting excessive gradients. Learning rate also matters, and Adam’s adaptive update is not identical to the raw clipped gradient, so clipping alone is not a universal bound on every parameter change. **Answer: C — max_grad_norm = 1.0.** A: **Incorrect:** Batch size changes how many examples contribute to a step, rather than directly clipping gradient magnitude. B: **Incorrect:** Gradient accumulation combines microbatches but does not impose a norm ceiling. C: **Correct:** max_grad_norm=1.0 activates the direct gradient-norm clipping mechanism. D: **Incorrect:** Saving checkpoints each epoch affects persistence rather than gradient magnitude.

**1. Build the initial dataset:** Dataset.from_dict creates integer id values 1,2,3 and string text values “Review one”, “Review two”, “Review three”. These are the two initial columns. **2. Dry-run the mapping function:** The substring “one” is present in “Review one”, so that row receives sentiment “positive”. The other two rows receive “neutral”. The function returns each example with its original fields plus the new sentiment field. **3. Infer the schema:** map preserves existing columns unless removal is requested and can add fields returned by the function. The resulting features are id:int64, text:string and sentiment:string. **Answer: C — id: int64; text: string; sentiment: string.** A: **Incorrect:** It describes only the original two columns and omits the newly returned sentiment field. B: **Incorrect:** The existing id and text columns are retained; the mapping does not return a sentiment-only dataset. C: **Correct:** It includes both original columns and the new string-valued sentiment column. D: **Incorrect:** Dataset.map can add columns by returning new fields; this mapping is valid.

**1. Describe the update:** LoRA freezes a pretrained matrix \(W\) and learns a low-rank correction \(\Delta W=BA\). The effective matrix is \(W+BA\). **2. Compare trainable counts:** If \(W\) has shape \(m\times n\) and the chosen rank r is small, the correction needs \(r(m+n)\) trainable values instead of updating all \(mn\) base values. **3. State the implication:** This reduces trainable-parameter, gradient and optimizer-state requirements. The pretrained model still needs to be available, and successful adaptation still requires suitable training data and evaluation. **Answer: B — Only a small number of new parameters are trained while freezing original .** A: **Incorrect:** LoRA does not guarantee a tenfold inference speedup. B: **Correct:** It trains a small low-rank update while freezing the original pretrained weights. C: **Incorrect:** Parameter efficiency does not remove the need for suitable data or a learning signal. D: **Incorrect:** LoRA adapts an existing pretrained model rather than retraining it from scratch.

**1. Write the intended score:** The commonly taught WordPiece pair score is \(\operatorname{score}(A,B)=\operatorname{count}(A,B)/[\operatorname{count}(A)\operatorname{count}(B)]\). **2. Explain the normalization:** A frequent pair is less distinctive if its individual components are already extremely frequent. Dividing by their separate counts favors pairs that occur together strongly relative to those marginal counts. **3. Contrast with plain BPE:** Frequency-only BPE chooses a most frequent pair. The normalized association criterion can select a less frequent pair with a stronger relative association. It is based on corpus statistics rather than semantic embeddings. **Answer: B — count(A,B) / [count(A) × count(B)].** **Convention note:** This is the standard pedagogical reconstruction of WordPiece training used by the question; exact implementations and tokenizer-training conventions can differ. A: **Incorrect:** This is the reciprocal of the intended association score, reversing its preference when maximized. B: **Correct:** The pair count is normalized by the product of its component counts, matching the stated WordPiece criterion. C: **Incorrect:** The score uses occurrence statistics, not an independently computed semantic-similarity measure. D: **Incorrect:** A raw pair count omits the normalization that distinguishes the intended criterion from BPE.

**1. Factor the sequence probability:** Causal language modeling uses \(P(x_1,\ldots,x_T)=\prod_{t=1}^TP(x_t\mid x_{<t})\). Training maximizes the sum of these next-token log probabilities. **2. Prevent future-token leakage:** In a GPT-style self-attention model, a causal mask blocks access to tokens after the current prediction position. Training can evaluate many positions in parallel while respecting this restriction. **3. Match the architecture and task:** This is the autoregressive generation objective associated with decoder-only GPT models. Random masking with bidirectional context is a different objective. **Answer: B, C, E — Auto-regressive next-token prediction.; Requires causal attention mask.; Maximizes joint probability of sequence..** A: **Incorrect:** Randomly masked-token reconstruction describes MLM rather than GPT’s next-token objective. B: **Correct:** Each token is predicted from its preceding context, which is autoregressive next-token prediction. C: **Correct:** Causal masking prevents future positions from leaking into GPT-style self-attention predictions. D: **Incorrect:** The standard match is decoder-only causal generation rather than a bidirectional encoder-only model. E: **Correct:** Maximizing the factorized next-token probabilities maximizes the autoregressive joint likelihood.

**1. Identify the constraints:** A 175B-parameter model is expensive to update in full, and 500 labels do not justify pretraining a model from scratch. The proposed approach must reduce adaptation cost. **2. Use prompting where possible:** Zero-shot prompting gives a task instruction without examples. Few-shot in-context learning includes a small number of demonstrations in the prompt; neither requires gradient-based weight updates. **3. Use a small trainable update if training is feasible:** LoRA learns low-rank adapters while retaining frozen base weights, greatly reducing optimizer and gradient storage. The base model still needs to be hosted or distributed appropriately. **Answer: B, C, D — Zero-shot prompting; Few-shot in-context learning; LoRA (PEFT).** **Convention note:** Efficient adaptation does not mean the 175B base model can fit on any single small GPU. The case assumes access to the model; hosting/inference still carries substantial costs. A: **Incorrect:** Updating all 175B parameters is the expensive approach the limited-GPU setting is intended to avoid. B: **Correct:** Zero-shot instructions can use the existing model without training updates, though quality must be evaluated. C: **Correct:** A few demonstrations in the context can adapt behavior without weight updates. D: **Correct:** LoRA reduces the trainable parameter count and is a standard efficient adaptation option. E: **Incorrect:** Pretraining a 175B-scale model is far more resource-intensive and cannot be supported by the small labeled dataset alone.

**1. Identify the early stage:** Statistical n-gram systems model short token histories and traditionally require substantial task-specific modeling. Their context and transfer capabilities are limited relative to later pretrained models. **2. Identify transfer learning:** Models such as BERT and GPT are pretrained on a general language objective and then adapted to downstream tasks. Reusing learned representations reduces the need to build a separate model from scratch for every task. **3. Identify in-context behavior:** Large pretrained models can perform some tasks from instructions or examples in their input without changing weights. Transformers enable highly parallel training through attention rather than adding recurrent connections. **Answer: A, B, C — The earliest models (e.g., n-grams) were statistical, required task-specific ; The ”pre-train, fine-tune” era (e.g., BERT, GPT) introduced transfer learning, ; Modern Large Language Models (LLMs like GPT-4) exhibit ”emerging abilities,” .** A: **Correct:** The statement correctly describes the broad statistical, task-specific phase, while summarizing a diverse history. B: **Correct:** General pretraining followed by downstream adaptation is the pretrain–fine-tune paradigm. C: **Correct:** Zero- and few-shot in-context task performance can occur without a gradient update to model weights. D: **Incorrect:** Word2vec learns word representations; it is not a complete GPT-like autoregressive text generator. E: **Incorrect:** Transformers remove the dependence on sequential recurrent state updates in their standard architecture rather than introducing recurrence.

**1. Match bidirectional encoding:** Encoder-only models such as BERT build context-sensitive representations from both directions and are often pretrained by masked-token reconstruction. They are widely used for understanding and classification. **2. Match causal generation:** Decoder-only GPT models predict the next token from preceding tokens and generate continuations autoregressively. **3. Match conditional sequence generation:** Encoder–decoder models encode a source sequence and generate a target sequence, making them natural for translation and summarization. Their pretraining can use denoising or other sequence-to-sequence objectives rather than one universal CLM objective. **Answer: A, B, C — Encoder-only models are best for natural language understanding tasks (like ; Decoder-only models are ideal for text generation tasks and are pre-trained ; Encoder-Decoder models are most suitable for sequence-to-sequence tasks .** A: **Correct:** Encoder-only representations and masked-language pretraining are a standard match for understanding tasks. B: **Correct:** Decoder-only causal language modeling naturally supports autoregressive text generation. C: **Correct:** An encoded source plus generated target fits sequence-to-sequence tasks. D: **Incorrect:** Different families use different objectives; BERT’s MLM and common T5/BART denoising objectives are not identical to GPT CLM.

**1. Count parameter storage:** \(600\) million values at four bytes each require \(600\times10^6(4)=2.4\times10^9\) bytes. **2. Count gradient storage:** Full fine-tuning stores one same-sized gradient for each parameter. This contributes another \(2.4\times10^9\) bytes. **3. Add and convert:** The requested subtotal is \(4.8\times10^9\) bytes, or 4.8 GB using the question’s decimal definition \(1\text{ GB}=10^9\) bytes. Optimizer states and activations are deliberately outside this subtotal. **Answer: 4.8.**