Deep Learning Practice · Quiz 1 · Jan 2025

**1. Follow the return value:** tokenizer.encode(text) returns an encoding object. Its tokens field contains token strings, whereas its ids field contains the corresponding integer vocabulary identifiers. **2. Apply the requested attribute:** The code accesses .ids, so its output must be a list of integers. Among the choices, only [102, 463, 509, 101, 2020] has that form. **Answer: B — [102, 463, 509, 101, 2020].** **Convention note:** The trained vocabulary is not supplied, so these exact integers cannot be derived from the sentence. B is the intended likely output because its type is correct, not because those numbers are universal token IDs. A: **Incorrect:** This is a list of text tokens, which corresponds to .tokens rather than .ids. B: **Correct:** This is the only integer-ID list, matching the requested encoding attribute. C: **Incorrect:** These are strings, including an empty string; they are not integer vocabulary IDs. D: **Incorrect:** This is the original input string, not the IDs returned by encoding.

**1. Understand BPE:** Byte-pair encoding begins with smaller symbols and learns frequent adjacent merges from training data. A new word can therefore be represented using multiple known subwords instead of requiring a whole-word vocabulary entry. **2. Inspect the training data:** The examples are existent, non, exist and word. With a vocabulary limit of 5000 and this tiny corpus, the learned merges include non, existent and word. **3. Tokenize the unseen concatenation:** nonexistentword can be split into those three known pieces: non + existent + word. All its characters are seen in training, so absence of an unknown-token setting does not cause an error for this input. **Answer: C — The tokenizer will output "non", "existent", "word" as separate tokens since it .** A: **Incorrect:** An unseen whole word need not become an unknown token when known subwords cover it. B: **Incorrect:** The full concatenated word does not occur in training and is not learned as one token. C: **Correct:** The learned subwords cover the input as non, existent and word. D: **Incorrect:** The learned merges form existent as a longer token, so this is not the resulting segmentation. E: **Incorrect:** No unseen character requires an unknown fallback here; the input can be encoded using learned pieces.

**1. Explain the adapter:** LoRA learns a small low-rank update to selected pretrained weight matrices. Instead of training every entry of a full matrix W, it learns factors A and B so that the effective update has the form \[W_{\mathrm{effective}}=W+\frac{\alpha}{r}BA.\] Here r is the rank and alpha controls scaling. **2. Apply the PEFT wrapper:** The base model already exists and lora_config describes the adapter. get_peft_model(base_model, lora_config) installs the configured adapter and returns the PEFT model ready for subsequent training. **Answer: D — get_peft_model(base_model, lora_config).** A: **Incorrect:** add_lora_adapter is not the indicated standard Transformers/PEFT method. B: **Incorrect:** from_config expects a model architecture configuration; a LoRA configuration does not create the pretrained classification model. C: **Incorrect:** The low-level LoraModel constructor also requires an adapter name in the relevant PEFT API; the two-argument call shown is incomplete. D: **Correct:** get_peft_model is the documented wrapper for an existing base model and PEFT configuration.

**1. Separate setup from execution:** TrainingArguments configures such matters as output paths and evaluation/checkpoint timing. Constructing Trainer connects the model, datasets and those settings; construction alone does not start optimization. **2. Start the training loop:** Calling trainer.train() runs the training procedure, including batch loading, loss computation, backpropagation and optimizer updates for trainable parameters. **3. Distinguish model mode:** A PyTorch model’s train() method switches modules such as dropout into training mode. It does not by itself loop over a dataset or update weights. **Answer: B — trainer.train().** A: **Incorrect:** Trainer does not expose the shown fine_tune() entry point. B: **Correct:** Trainer.train() starts its configured optimization loop. C: **Incorrect:** peft_model.train() changes module mode but does not perform fine-tuning on its own. D: **Incorrect:** The displayed PEFT model is not a Keras model with the shown fit() training API.

**1. Explain the vocabulary trade-off:** A subword vocabulary stores reusable token pieces. A larger vocabulary can include more long words and longer pieces; a smaller one generally needs more pieces to represent the same text. **2. Apply the likely comparison:** For a word such as unhappiness, a 5K tokenizer may split it into several short tokens while a 50K tokenizer may have a longer merged representation. Thus the intended likely trend is \[n_{t,5K}>n_{t,50K}.\] **3. Separate a trend from a guarantee:** Vocabulary sizes alone do not determine the exact segmentation of one word. The training corpus and learned merges are also needed. **Answer: A — **n_t_5k > n_t_50k** because with a smaller vocabulary size, the tokenizer will .** **Convention note:** A is a likely trend, not a strict theorem for this unspecified corpus. Both tokenizers could encode this particular word with the same count. The code also trains a second time on get_examples(), so its final merges depend on that supplied iterator. A: **Correct:** This is the intended typical trend: smaller vocabularies often produce longer token sequences. B: **Incorrect:** Having fewer stored pieces does not imply fewer pieces are needed to encode a word. C: **Incorrect:** Token count depends on the vocabulary and its learned merges, not only on the input word. D: **Incorrect:** Subword tokenization does not guarantee identical segmentation at different vocabulary sizes.

**1. Define causal language modeling:** The model predicts each next token using only the preceding tokens. Its left-to-right factorization is \[P(x_1,\ldots,x_T)=\prod_{t=1}^{T}P(x_t\mid x_1,\ldots,x_{t-1}).\] **2. State the training objective:** GPT-2 minimizes the negative log-likelihood of the actual next tokens, \[\mathcal L=-\sum_t\log P(x_t\mid x_{<t}).\] A causal attention mask prevents a position from using future tokens to predict its target. **Answer: C — Causal Language Modeling (CLM).** A: **Incorrect:** Masked language modeling reconstructs selected masked positions using context on both sides; that is not GPT-2’s original objective. B: **Incorrect:** Encoder-decoder sequence-to-sequence modeling is not GPT-2’s original decoder-only pretraining objective. C: **Correct:** Next-token prediction with a causal mask is causal language modeling. D: **Incorrect:** Text classification predicts a category rather than the next token at every position.

**1. Define gradient-based tuning:** Fine-tuning computes a loss on training examples, differentiates it with respect to trainable parameters, and updates those parameters using an optimizer. **2. Identify parameter-updating methods:** Instruction fine-tuning trains on instruction-response pairs. Parameter-efficient fine-tuning updates a limited set of parameters, such as LoRA factors. RLHF preference tuning uses human-preference information to guide optimization, commonly through gradient-based reward-model and policy training. **3. Distinguish prompting:** In the terminology intended here, zero-shot and few-shot refer to providing zero or a few examples in the prompt. They change the input context while leaving model parameters fixed. **Answer: C, D, E — Instruction Fine tuning; Parameter Efficient Fine Tuning; Preference Tuning Via RLHF.** **Convention note:** Zero-shot and few-shot are interpreted as prompting in this question. Few-shot learning can also involve actual gradient-based fine-tuning in other contexts; the label alone does not rule that out. A: **Incorrect:** Ordinary zero-shot prompting performs no gradient update. B: **Incorrect:** Ordinary few-shot in-context prompting supplies examples without updating parameters. C: **Correct:** Instruction-response training computes gradients and changes trainable parameters. D: **Correct:** Updating fewer parameters still requires gradients; parameter efficiency does not mean absence of training. E: **Correct:** RLHF uses preference information to train reward/policy components through gradient-based optimization.

**1. Understand Dataset.map:** With its default unbatched operation, map calls the function on each example dictionary. The returned dictionary supplies new or replacement column values. **2. Compute the character count:** For each row, len(example["text"]) counts characters in that row’s string. Returning {"char_count": ...} adds just the new field; returning the example after setting that field also supplies the required value. **3. Retain the result:** Dataset transformations return a new dataset, so assigning dataset = dataset.map(...) retains the added column. **Answer: A, C — Both dictionary-returning Dataset.map implementations.** A: **Correct:** The returned dictionary has the new char_count key and the correct character count for each row. B: **Incorrect:** This does not use map, and a Hugging Face text column is not a pandas Series with the shown apply method. C: **Correct:** The function adds char_count to the example dictionary and returns it; map incorporates that field. D: **Incorrect:** This is not the shown Dataset.map pattern; a column is not another Dataset with that row-mapping API.

**1. Expand the index sequence:** range(0, 500, 10) generates 0, 10, 20, ..., 490. The stop value 500 is excluded. **2. Count the selected rows:** \[\text{count}=\frac{490-0}{10}+1=50.\] Dataset.select uses these indices and returns a new Dataset containing the selected rows. **3. Track the objects:** The new object is assigned to sample. The original dataset is retained; selecting rows does not remove rows from it in place. **Answer: A, B, D — 50 samples; step size 10; result is a Dataset.** A: **Correct:** The generated range has exactly 50 indices. B: **Correct:** Successive selected row indices differ by 10. C: **Incorrect:** select returns a new object; the original dataset is not modified in place. D: **Correct:** Selecting rows from a Dataset returns a Dataset.

**1. Compute the selected split size:** The full training split has 25,000 samples. train[:25%] keeps one quarter, so the intermediate dataset has \[25000\times0.25=6250\text{ samples}.\] **2. Apply the length filter:** The filter keeps text strings with length greater than 200. By the stated uniform-distribution assumption, 60% of the selected split passes this condition. **3. Calculate the final count:** \[6250\times0.60=3750\text{ samples}.\] The percentage is applied after taking the quarter split, not to all 25,000 rows. **Answer: 3750.**

**1. Read the relevant configuration:** The embedding width is d = 1024 and there are h = 16 attention heads. Each head has width \(d_h=d/h=64\). The question asks for the query, key and value weight matrices in one transformer block. **2. Count one projection:** Across all heads, a query projection maps a 1024-dimensional vector to 1024 projected components. Thus \(W_Q\) contains \(1024\times1024=1048576\) weights. Equivalently, the heads contribute \(16\times1024\times64\) weights. **3. Include all three projections:** Key and value projections have the same shape. Hence \[N_{QKV}=3d^2=3(1024^2)=3145728.\] This count is for one block, so it is not multiplied by the 24-layer count. **Answer: 3145728.** **Convention note:** The count includes the three weight matrices only. Including their bias vectors would add 3 × 1024 = 3072 parameters, giving 3,148,800.

**1. Identify the token embedding table:** The vocabulary size is V = 50,257 and each token has an embedding with d = 1024 components. The table therefore has V rows and d columns. **2. Count its trainable entries:** \[N_{\mathrm{token}}=Vd=50257\times1024=51463168.\] **3. Convert to millions:** \[\frac{51463168}{10^6}=51.463168\text{ million}\approx51.46\text{ million}.\] Round only after the full multiplication. **Answer: 51.46.** **Convention note:** The scoring range indicates token embeddings only. GPT-2 also has 1024 × 1024 = 1,048,576 learned positional-embedding entries. Counting both token and position embeddings would give 52.51 million, so the distinction matters.

**1. Explain the context window:** A causal transformer processes a bounded sequence of token positions. Its context setting describes how many tokens can be included in that window, not how many attention heads or embedding coordinates exist. **2. Read this configuration:** n_ctx is 1024 in the displayed GPT-2 configuration. In the intended configuration terminology, it represents the maximum context/input length in tokens. The separate n_head and n_embd fields describe heads and embedding width. **Answer: B — The maximum length of the input sequence in tokens..** **Convention note:** In newer Transformers implementations, n_positions/max_position_embeddings is the operative positional-capacity field and n_ctx can be a legacy configuration attribute. The answer uses the context-length meaning intended by this exam. A: **Incorrect:** Attention-head count is specified by n_head, here 16. B: **Correct:** The field represents the token context length, here 1024. C: **Incorrect:** Embedding width is specified by n_embd, here 1024; the equal numerical value does not make the meanings identical. D: **Incorrect:** The total parameter count is derived from many configuration fields; n_ctx is not that count.