Questions and published explanations below are available without starting a test. Some questions may not have a published solution yet.
14 / 14 published solutions
Question 2 MCQ · 2.0 marks
Which among the following is true for a McCulloch-Pitts (MP) neuron?
6406536373443.
It can implement Boolean functions with a non-linear decision boundary.
6406536373444.
It can implement linearly separable Boolean functions with a linear decision
boundary.
6406536373445.
It can be used to approximate a real-valued function.
6406536373446.
It can accurately represent the XOR function by adjusting its weights and
thresholds.
Published solution
**1. Write the neuron rule:** An MP neuron forms a weighted sum and applies a hard threshold, giving a binary output: \(y=\mathbf1\{w^Tx\ge\theta\}\).
**2. Identify its expressive limit:** The boundary \(w^Tx=\theta\) is a hyperplane. A single such neuron can represent linearly separable Boolean functions, but not XOR or arbitrary real-valued outputs.
**Answer: B — It can implement linearly separable Boolean functions with a linear decision .**
A: **Incorrect:** A single threshold neuron has a linear, not a general nonlinear, decision boundary.
B: **Correct:** A threshold hyperplane separates its two output classes.
C: **Incorrect:** Its output is binary; it is not a general real-valued function approximator.
D: **Incorrect:** The alternating labels of XOR cannot be separated by a single line.
Consider a shallow neural network with a scalar input
and a hidden layer with three neurons.
The outputs of the hidden layer neurons are given by
where
denotes the activation function. The final output of the network is given by
Suppose we multiply the parameters
and
by a constant
, and divide
by the
same constant
, while all other parameters remain unchanged. Which of the following
statements is correct?
6406536373447.
The network output
remains unchanged if the activation function used is
ReLU for
,
.
6406536373448.
The network output
remains unchanged if the activation function used is
ReLU for
,
.
6406536373449.
The network output will be unchanged if the activation function used is
sigmoid for
.
6406536373450.
After the transformation, the contribution
to the output remains
unchanged for all
and for all
.
Published solution
**1. Isolate the changed path:** Put \(z=b_{11}+w_{11}x\). Its original contribution is \(w_{21}a(z)\); after rescaling it is \((w_{21}/\alpha)a(\alpha z)\).
**2. Apply positive homogeneity:** For \(\alpha>0\), \(\operatorname{ReLU}(\alpha z)=\alpha\operatorname{ReLU}(z)\), so the factor cancels and every other path is unchanged.
**3. Test the alternatives:** Negative scaling switches the active half-line. Sigmoid is not homogeneous: at \(z=0,\alpha=2\), its rescaled contribution is \(w_{21}/4\), versus \(w_{21}/2\) originally.
**Answer: A — ReLU with positive scaling preserves the output.**
A: **Correct:** Positive ReLU homogeneity exactly cancels the reciprocal output-weight rescaling.
B: **Incorrect:** For z=1 and alpha=-1, ReLU changes from 1 to 0, so cancellation fails.
C: **Incorrect:** Sigmoid does not satisfy a(alpha z)=alpha a(z).
D: **Incorrect:** The claimed invariance for all nonzero alpha fails for negative ReLU scaling and for general activations.
Consider a multilayer perceptron (MLP) with two binary inputs
and
, 2 neurons in the hidden
layer, and 1 in the output layer. Let
and
be the outputs of the hidden layer and it computes
The output of the network is given by
, where
Which of the following are the possible choices of
for the network to correctly compute the
XOR function, given that
.
6406536373451.
0.5
6406536373452.
-0.55
6406536373453.
-0.60
6406536373454.
1
Published solution
**1. Evaluate the hidden units:** \(h_1=\operatorname{step}(x_1+x_2-0.5)\) is OR and \(h_2=\operatorname{step}(x_1+x_2-1.5)\) is AND. Their outputs for 00,01,10,11 are respectively (0,0),(1,0),(1,0),(1,1).
**2. Enforce XOR:** The output scores are \(-0.5,0.5,0.5,0.5+w_2\). Since step returns 1 at zero, the last score must be strictly negative: \(w_2<-0.5\).
**3. Select allowable weights:** Both -0.55 and -0.60 satisfy this inequality; 0.5 and 1 do not.
**Answer: B, C — -0.55; -0.60.**
A: **Incorrect:** At input 11 the score is 1, giving 1 instead of XOR’s 0.
B: **Correct:** At input 11 the score is -0.05, while single-one inputs still score 0.5.
C: **Correct:** At input 11 the score is -0.10, with all other outputs correct.
D: **Incorrect:** At input 11 the score is 1.5, giving the wrong output.
Consider a neural network for a regression problem with input
. The network has
hidden layers, each with
sigmoid neurons. The output layer has
neuron and uses the sigmoid
activation function. Enter the number of parameters in the network.
Published solution
**1. Count the first hidden layer:** Ten inputs feed five neurons. Weights plus biases give \(10\times5+5=55\).
**2. Count subsequent layers:** The second five-neuron layer has \(5\times5+5=30\) parameters. The single output has \(5\times1+1=6\).
**3. Add all trainable quantities:** \(55+30+6=91\). Sigmoid itself has no trainable parameters.
**Answer: 91.**
Consider a training set consisting of 12 samples. The vanilla gradient descent algorithm is used to
update the model parameters over 4 epochs. The learning rate follows an exponential decay
scheme given by
where the iteration index
starts from zero and increments only when the parameters are
updated. What is the value of the learning rate
at the end of training?
(Note: Enter your answer up to 3 decimal points. For example, if your answer is 0.12345, enter 0.123).
Published solution
**1. Count updates, not examples:** Vanilla/full-batch gradient descent makes one parameter update per epoch. Four epochs therefore use indices \(t=0,1,2,3\); the 12 examples do not make 12 updates per epoch.
**2. Evaluate the final training-step rate:** \(\eta_3=0.8e^{-0.2(3)}=0.439049\ldots\), giving 0.439 to three decimals.
**Answer: 0.439.**
**Convention note:** “At the end” is interpreted as the learning rate used for the final update, matching the supplied range. The next scheduled rate after completing four updates would be eta_4=0.8 exp(-0.8)=0.359463..., but that rate is not used during these four epochs.
A content recommendation system models whether a user will engage with an item based on
features
.
The training dataset is
where
denotes non-engagement (0) or engagement (1). The probability of
engagement is modeled using:
where
, and
denote the bias. The model parameters are learned using gradient descent
by minimizing the squared error loss:
Consider a training example that takes in an input vector , with true label
. The
weight vector
is initialized to and
. Compute the loss after one update of
and
for
. Enter the answer correct to two decimal places.
Published solution
**1. Perform the initial forward pass:** With \(w=(0,0)^T,b=0,x=(1,1)^T\), the logit is zero and \(\hat y=0.5\). For this one-example update, \(L=(1-\hat y)^2=0.25\).
**2. Differentiate and update:** \(\partial L/\partial z=2(\hat y-y)\hat y(1-\hat y)=-0.25\). Thus \(\nabla_wL=(-0.25,-0.25)^T\) and \(\partial L/\partial b=-0.25\). With learning rate 1, \(w^+=(0.25,0.25)^T,b^+=0.25\).
**3. Recompute the loss:** The new logit is 0.75, so \(\hat y^+=\sigma(0.75)=0.6791787\). Hence \(L^+=(1-0.6791787)^2=0.1029263\), rounded to 0.10.
**Answer: 0.10.**
**Convention note:** The supplied individual training example is treated as the one-example loss/update. A full-dataset update would require all examples, which are not supplied.
Consider the truth table of a NAND function for two binary inputs:
You use a perceptron for implementing the NAND function. The output of the neuron is given as
is assumed to be 1.
Based on the above data, answer the given subquestions.
Is the function linearly separable?
6406536373458.
Yes
6406536373459.
No
Published solution
**1. Separate the lone negative point:** NAND is 0 only at (1,1), and 1 at (0,0),(0,1),(1,0).
**2. Give an explicit separator:** \(s=1-x_1-x_2\) has scores 1,0,0,-1 in that order. With the stated rule \(y=1\) when \(s\ge0\), these are exactly the NAND labels.
**Answer: A — Yes.**
A: **Correct:** The explicit linear threshold 1-x1-x2 implements the truth table.
B: **Incorrect:** A separating line exists, so nonseparability is false.
**1. Translate the truth table into inequalities:** NAND requires \(w_0\ge0\), \(w_0+w_1\ge0\), \(w_0+w_2\ge0\), and \(w_0+w_1+w_2<0\). The last inequality is strict because a zero score predicts 1.
**2. Check every proposed vector:** Scores for 00,01,10,11 are A: (2,1,1,0); B: (1,0,0,-1); C: (-2,-3,-3,-4); D: (0.5,0.25,0.25,0). Only B gives outputs (1,1,1,0).
**Answer: B — Weights (w0,w1,w2)=(1,-1,-1).**
A: **Incorrect:** The 11 score is zero, predicting 1 rather than 0.
B: **Correct:** Its four scores produce the exact NAND truth table.
C: **Incorrect:** Even input 00 is wrongly predicted 0 because its score is negative.
D: **Incorrect:** The 11 score is zero and therefore predicts 1.
**1. Start with zero weights:** Use the usual unit-rate mistake update \(w\leftarrow w+yx\). At (2,1), the score is 0 and the stated nonnegative rule predicts +1 correctly; no update occurs.
**2. Process the remaining examples:** At (-1,-2), the score is 0 but the label is -1, so \(w=(0,0)+(-1)(-1,-2)=(1,2)\). At (1,3), the score is 7, correctly positive. At (-2,-1), the score is -4, correctly negative.
**3. Check the final model:** With \(w=(1,2)^T\), the four scores are 4,-5,7,-4. All have the correct labels, so another pass makes no changes.
**Answer: B — Final weight vector (1,2) transposed.**
A: **Incorrect:** This results from updating the first zero-score positive example unnecessarily.
B: **Correct:** Only the second sample causes a mistake update, producing (1,2).
C: **Incorrect:** No valid mistake update in this pass produces (1,3).
D: **Incorrect:** This points toward the negative examples and is not the traced update.
Suppose you perform the perceptron algorithm on the following dataset:
Assume the initial weights
and we pass the data points in the order given in the
table. The following rule is used for the classification:
Based on the above data, answer the given subquestions.
After one epoch, your friend claims that the perceptron weights no longer need updating, i.e., the
model has converged. Specify whether the statement is true or false.
6406536373468.
True
6406536373469.
False
Published solution
**1. Start with zero weights:** Use the usual unit-rate mistake update \(w\leftarrow w+yx\). At (2,1), the score is 0 and the stated nonnegative rule predicts +1 correctly; no update occurs.
**2. Process the remaining examples:** At (-1,-2), the score is 0 but the label is -1, so \(w=(0,0)+(-1)(-1,-2)=(1,2)\). At (1,3), the score is 7, correctly positive. At (-2,-1), the score is -4, correctly negative.
**3. Check the final model:** With \(w=(1,2)^T\), the four scores are 4,-5,7,-4. All have the correct labels, so another pass makes no changes.
**Answer: A — True.**
A: **Correct:** The final scores 4,-5,7,-4 classify every sample correctly, so the model is already stable.
B: **Incorrect:** There is no remaining misclassified sample requiring an update.
Consider a neural network for a binary classification problem with one input and one output.
There is one hidden layer with two neurons and an output layer. Both the layers uses sigmoid as
an activation function. Cross entropy loss is used.
Following are the values of weights and biases:
Based on the above data, answer the given subquestions.
Compute the predicted output
for the given input
. Enter the answer correct to two decimal
places.
Published solution
**1. Compute hidden activations:** At x=1, \(h_1=\sigma(1)=0.7310586\) and \(h_2=\sigma(-1)=0.2689414\). They sum to 1.
**2. Compute the output:** \(z=h_1+h_2+1=2\), hence \(\hat y=\sigma(2)=0.8807971\), or 0.88.
**Answer: 0.88.**
Consider a neural network for a binary classification problem with one input and one output.
There is one hidden layer with two neurons and an output layer. Both the layers uses sigmoid as
an activation function. Cross entropy loss is used.
Following are the values of weights and biases:
Based on the above data, answer the given subquestions.
Compute
.
Enter the answer correct to two decimal places.
Published solution
**1. Compute hidden activations:** At x=1, \(h_1=\sigma(1)=0.7310586\) and \(h_2=\sigma(-1)=0.2689414\). They sum to 1.
**2. Compute the output:** \(z=h_1+h_2+1=2\), hence \(\hat y=\sigma(2)=0.8807971\), or 0.88.
**3. Backpropagate cross-entropy:** For sigmoid output with binary cross-entropy, \(\partial L/\partial z=\hat y-y\). Therefore \(\partial L/\partial w_1=(\hat y-y)v_1h_1(1-h_1)x\).
**4. Substitute values:** \((0.8807971-1)(1)(0.7310586)(0.2689414)(1)=-0.0234367\ldots\), which rounds to -0.02.
**Answer: -0.02.**
**1. Differentiate the averaged loss:** \(J(w)=\frac1{2m}\sum_i(wx_i-y_i)^2\), so \(J\prime(w)=\frac1m\sum_i(wx_i-y_i)x_i\).
**2. Evaluate the first batch:** At w=1, (x,y)=(1,2) contributes -1 and (2,4) contributes -4. Their average is \((-1-4)/2=-2.5\).
**Answer: A — Gradient -2.5.**
A: **Correct:** The averaged gradient is (-1-4)/2=-2.5.
B: **Incorrect:** This is the unaveraged sum; the loss includes division by m.
C: **Incorrect:** This is neither the sum nor the mean of the two contributions.
D: **Incorrect:** The sign is wrong, and averaging is omitted.
trained using momentum-based mini-batch gradient
descent with the loss function:
where
is the batch size. The algorithm uses a mini-batch size of
, learning rate
, and
the momentum coefficient
. The weights and velocity are initialized as
and
. The update rules are given by:
The first mini-batch consists of the following samples:
Based on the above data, answer the given subquestions.
Using the gradient computed in the previous question, determine the velocity
after the first
iteration.
Enter the answer correct to one decimal place.
Published solution
**1. Differentiate the averaged loss:** \(J(w)=\frac1{2m}\sum_i(wx_i-y_i)^2\), so \(J\prime(w)=\frac1m\sum_i(wx_i-y_i)x_i\).
**2. Evaluate the first batch:** At w=1, (x,y)=(1,2) contributes -1 and (2,4) contributes -4. Their average is \((-1-4)/2=-2.5\).
**3. Apply momentum:** The printed rule gives \(u_0=\beta u_{-1}+J\prime(w_0)=0.9(0)-2.5=-2.5\). The learning rate multiplies velocity only when updating w; it is not part of this velocity definition.
**Answer: -2.5.**
**Convention note:** The source uses w0 with a time index but w_(t+1) in the update. Following the explicitly supplied initial values and velocity rule, the first velocity is -2.5; the subsequent weight would be 1.25.
