Large Language Models · End Term · May 2026 · AN

Consider the following Assertion (A) and Reason (R) about the Transformer encoder. **Assertion (A):** In a Transformer encoder, each token can attend to every other token in the input sequence through self-attention. **Reason (R):** The encoder's self-attention mechanism uses a causal mask to prevent each token from attending to tokens that appear later in the sequence. Choose the correct option:

Which of the following statements regarding one-hot positional encoding and sinusoidal positional encoding is incorrect?

You are designing an LLM to translate a sentence from English to Spanish with low computational complexity. The system has to consider a few possible sequences of words rather than committing to the single highest-probability word at every step. Which of the following decoding strategies is the most appropriate?

In the Next Sentence Prediction task, how are the two input sentences distinguished by the BERT architecture?

Consider a mini-batch of 3 samples, each with 2 features. The input matrix X is given below, where rows represent individual samples and columns represent features:

Layer Normalization is applied on this with a learned scale parameter

and a shift parameter

. What is the final output value of the first feature for the third sample?

Consider a tokenizer processing the word "lion". The vocabulary contains the following tokens along with the assigned log-probabilities.

To determine the optimal tokenization, the tokenizer uses the forward decoding algorithm (Viterbi approach). Determine the most likely tokenization for the word "lion" using the algorithm. Instructions: Enter your answer as a lowercase string with the segments separated by a comma. For example, if the word is whereby and the answer has two segments where and by, enter the answer as **where,by** **NOTE:** Enter the exact answer without any extra space in the beginning or at the end.

Suppose the relative position between two tokens at positions

and

is defined as

To limit the range of relative positions, clipping is applied using

where

. For a sequence of length

, **how many pairwise relative-position values are modified** **by the clipping operation?**

Consider an LLM operating on a vocabulary consisting of 5 tokens. For a given context, the model produces the following logits:

To obtain the next token during generation, the model uses top-p sampling with p = 0.80. Based on the above data, answer the given subquestions.

How many tokens are selected as candidate tokens?

Consider an LLM operating on a vocabulary consisting of 5 tokens. For a given context, the model produces the following logits:

To obtain the next token during generation, the model uses top-p sampling with p = 0.80. Based on the above data, answer the given subquestions.

After renormalization of the probabilities, what is the highest probability value among those? (Round off to three decimal places)

How did the GPT-2 model initially demonstrate "zero-shot domain transfer" capabilities on complex tasks like summarization and translation?

Match the following unsupervised objectives (I,II,III) to their corresponding Target output structures (A,B,C) based on the following: I. Prefix Language Modelling II. Masked Seq-to-Seq pretraining III. Random spans A. Interleaved sentinel tokens and missing text B. The entire original text sequence C. The suffix of the original text

Consider the original text "The quick brown fox jumps over the lazy dog". If the input for a prefix language modelling objective is "The quick brown fox", what will the target be?

Consider a transformer model that uses strided local attention with a fixed window size c. If the sequence length T is doubled, by what factor does the computational time complexity of the attention mechanism change?

What is the primary trade-off made by utilizing KV Cache during autoregressive decoding?

A Large Language model uses Grouped Query Attention with a KV cache during inference. Consider the following model configuration: • Precision: FP16 (16-bit floating point) • Number of Transformer layers : 32 • Number of Query attention heads : 32 • Number of Key attention heads : 8 • Number of Value attention heads : 8 • Head dimension : 128 • Batch size : 4 • Maximum sequence length per request : 1024 tokens **Assume 2 bytes per FP16 value and 1 MB = 1048576 bytes** **calculate the total memory required for the KV cache expressed in MB. Enter the value** **rounded to the nearest integer**

Which of the following statements about length generalization in Large Language Models (LLMs) are correct?

Consider the following matrices for Transformer architechture. The token representations at positions

and

are

The query and key projection matrices are

and

**RoPE Frequency** For RoPE, use the frequency formula

All angles are measured in **radians**. **RoPE Rotation Matrix** For position

, define the RoPE rotation matrix as

The position-dependent query and key vectors are

and

Therefore, their unnormalized attention score is

Using the property

the attention score can be expressed using the relative position:

where the relative position is

. For

calculate the unnormalized RoPE attention score

Based on the above data, answer the given subquestions.

Compute

and select the correct option

Consider the following matrices for Transformer architechture. The token representations at positions

and

are

The query and key projection matrices are

and

**RoPE Frequency** For RoPE, use the frequency formula

All angles are measured in **radians**. **RoPE Rotation Matrix** For position

, define the RoPE rotation matrix as

The position-dependent query and key vectors are

and

Therefore, their unnormalized attention score is

Using the property

the attention score can be expressed using the relative position:

where the relative position is

. For

calculate the unnormalized RoPE attention score

Based on the above data, answer the given subquestions.

Compute

and select the correct option

Consider the following matrices for Transformer architechture. The token representations at positions

and

are

The query and key projection matrices are

and

**RoPE Frequency** For RoPE, use the frequency formula

All angles are measured in **radians**. **RoPE Rotation Matrix** For position

, define the RoPE rotation matrix as

The position-dependent query and key vectors are

and

Therefore, their unnormalized attention score is

Using the property

the attention score can be expressed using the relative position:

where the relative position is

. For

calculate the unnormalized RoPE attention score

Based on the above data, answer the given subquestions.

Construct the corresponding RoPE rotation matrix (

) and submit the maximum value in it. (correct upto 2 decimal places) • All angles are measured in **radians**.

Consider the following matrices for Transformer architechture. The token representations at positions

and

are

The query and key projection matrices are

and

**RoPE Frequency** For RoPE, use the frequency formula

All angles are measured in **radians**. **RoPE Rotation Matrix** For position

, define the RoPE rotation matrix as

The position-dependent query and key vectors are

and

Therefore, their unnormalized attention score is

Using the property

the attention score can be expressed using the relative position:

where the relative position is

. For

calculate the unnormalized RoPE attention score

Based on the above data, answer the given subquestions.

Computing the final dot product

. (correct upto 2 decimal places)