Question 175 MCQ · 0.0 marks
Instructions has been mentioned above.
This Instructions is just for a reference & not for an evaluation.
A published solution is not available for this question yet.
Large Language Models · End Term · Sep 2024 · AN
Large Language Models · End Term · Sep 2024 AN
← Course papers · Start practice / exam
Questions and published explanations below are available without starting a test. Some questions may not have a published solution yet.
0 / 21 published solutions
Question 176 NAT · 3.0 marks
A published solution is not available for this question yet.
Question 177 MCQ · 3.0 marks
Assume that we have a large corpus of text. The vocabulary constructed from the text contains
10000 words. Of these, 100 words occurred only once in the entire corpus of text. The parameters
of the embedding layer and the output layer of the model are shared. Suppose we create a batch
of 256 samples (each sample is a sentence from the corpus). None of these samples contains any
of the 100 rare words. Suppose we pre-train the model for one iteration using the batch of
samples, then:
it is certain that the embeddings of none of these 100 rare words will get
updated.
there is a chance that the embeddings of all or some of these 100 rare words
will get updated
the embeddings of all these 100 rare words will defintely get updated
None of these
A published solution is not available for this question yet.
Question 178 MCQ · 3.0 marks
Suppose we use a pre-trained model for text generation with the given prompt “I am going to”.
Which of the following decoding strategies can be used such that the pre-trained model generates
same text completion each time it is executed
Beam search with beam size 4
Greedy approach
Top-K with k = 2
None of these
A published solution is not available for this question yet.
Question 179 MSQ · 3.0 marks
Assume that we have 12 sufficiently large supervised NLP datasets for different tasks like
sentiment classification, textual entailment, language understanding and so on. Suppose we take
all the samples (dropping labels) from these datasets to train the GPT model using the CLM
objective.Then which of the following transfer learning approaches is (are) appropriate to transfer
the knowledge to any of these 12 downstream tasks?
Zero shot learning
Few-shot learning
Full Fine-tuning
None of the given approaches are appropriate as the model has already been
trained using samples from all the 12 datasets
A published solution is not available for this question yet.
Question 180 MCQ · 2.0 marks
What is the order of the language modeling pipeline?
First, the model, which handles text and returns raw predictions.The tokenizer
then makes sense of these predictions and converts them back to text when needed.
First, the tokenizer, which handles text and returns IDs. The model handles
these IDs and outputs a prediction, which can be some text.
The tokenizer handles text and returns IDs. The model handles these IDs and
outputs a prediction. The tokenizer can then be used once again to convert these predictions back
to some text.
None of these
A published solution is not available for this question yet.
Question 181 MCQ · 2.0 marks
Consider following statment and mark if it is true or false:
Repeating examples in the pre-training datasets are completely harmless for downstream
performance.
TRUE
FALSE
A published solution is not available for this question yet.
Question 182 MCQ · 2.0 marks
What is the time complexity of the computing attention matrix A with strided local attention
(parameterized by c) and batch size of one? The symbols have the usual meaning.
A published solution is not available for this question yet.
Question 183 MCQ · 2.0 marks
k = 0 will perform same as compared to k = 600
k = 0 will perform worse compared to k = 600
k = 0 will perform better compared to k = 600
Value of k is irrelevant as the difference between position embeddings of two
consicutive tokens remain the same.
A published solution is not available for this question yet.
Question 184 MCQ · 2.0 marks
A published solution is not available for this question yet.
Question 185 NAT · 2.0 marks
How many tokens are there in the initial vocabulary?
A published solution is not available for this question yet.
Question 186 MCQ · 3.0 marks
Which of the following pairs has the least score before any merge?
A published solution is not available for this question yet.
Question 187 SHORT_TEXT · 3.0 marks
Which pair will be merged in the very first merge? Say the pair is (‘a’,‘b’), then enter **“ab”** (without
quotes and white spaces). If there is a tie between two or more candidates, pick the one that
occurs first in the original vocabulary.
**NOTE:** Enter the exact answer without any space in the beginning or at the end.
A published solution is not available for this question yet.
Question 188 NAT · 3.0 marks
What is the score of the pair merged in the very first merge?
A published solution is not available for this question yet.
Question 189 MCQ · 3.0 marks
A published solution is not available for this question yet.
Question 190 MSQ · 2.0 marks
Consider the following statements about decreasing the size of the vocabulary
It increases number of tokens required to represent a sentence
It Increases the computational complexity computing attention matrix
It increases the context length T
None of these
A published solution is not available for this question yet.
Question 191 MSQ · 2.0 marks
Choose the correct statements regarding an encoder-decoder transformer model:
Multi head masked attention block in decoder makes use of a mask.
Transformer model is an improvement over its RNN based predecessor for
machine translation.
For machine tranlation task the input and output embedding layers have to be
identical.
None of these.
A published solution is not available for this question yet.
Question 192 MSQ · 2.0 marks
Consider C4 pipeline. Which of the following will NOT pass through it as it is?
A published solution is not available for this question yet.
Question 193 MSQ · 2.0 marks
Which of the following are correct about GPT-1 model:
It is a decoder only model.
It uses absolute positional encoding.
It uses a composite dataset for pre-training.
None of these.
A published solution is not available for this question yet.
Question 194 NAT · 2.0 marks
How many entries of unnormalized attention matrix QK\(^{T}\) will be calculated with strided attention
mechanism with c = 5. The sequence length is T = 32
A published solution is not available for this question yet.
Question 195 NAT · 2.0 marks
How many non zero entries will be there in the mask matrix M?
A published solution is not available for this question yet.