You are evaluating a binary classifier. There are 50 positive outcomes in the test data, and 100 observations. Using a 50% threshold, the classifier predicts 40 positive outcomes, of which 10 are incorrect.Increasing the threshold to 60% results in 5 additional positive predictions, all of which are correct. Which of the following statements about this new model (compared with the original model that had a 50% threshold) is TRUE?0 / 1 pointThe F1 score of the classifier would decrease.The area under the ROC curve would decrease.The F1 score of the classifier would remain the same.The area under the ROC curve would remain the same.
Question
You are evaluating a binary classifier. There are 50 positive outcomes in the test data, and 100 observations. Using a 50% threshold, the classifier predicts 40 positive outcomes, of which 10 are incorrect.Increasing the threshold to 60% results in 5 additional positive predictions, all of which are correct. Which of the following statements about this new model (compared with the original model that had a 50% threshold) is TRUE?0 / 1 pointThe F1 score of the classifier would decrease.The area under the ROC curve would decrease.The F1 score of the classifier would remain the same.The area under the ROC curve would remain the same.
Solution
The F1 score of the classifier would increase.
Here's why:
The F1 score is the harmonic mean of precision and recall. Precision is the number of true positives divided by the total number of positive predictions, and recall is the number of true positives divided by the total number of actual positives.
In the original model with a 50% threshold, there were 40 positive predictions, of which 10 were incorrect. So, there were 30 true positives. The precision was 30/40 = 0.75, and the recall was 30/50 = 0.6. The F1 score was 2*(0.75*0.6)/(0.75+0.6) = 0.6667.
In the new model with a 60% threshold, there were 45 positive predictions, all of which were correct. So, there were 45 true positives. The precision was 45/45 = 1, and the recall was 45/50 = 0.9. The F1 score was 2*(1*0.9)/(1+0.9) = 0.9474.
So, the F1 score increased when the threshold was increased to 60%.
Similar Questions
You are evaluating a binary classifier. There are 50 positive outcomes in the test data, and 100 observations. Using a 50% threshold, the classifier predicts 40 positive outcomes, of which 10 are incorrect.The threshold is now increased further, to 70%. Which of the following statements is TRUE?1 pointThe Recall of the classifier would Increase.The Precision of the classifier would decrease.The Recall of the classifier would increase or remain the same.The Precision of the classifier would increase or remain the same.
You are evaluating a binary classifier. There are 50 positive outcomes in the test data, and 100 observations. Using a 50% threshold, the classifier predicts 40 positive outcomes, of which 10 are incorrect.What is the classifier’s Recall on the test sample?1 point25%60%75%80%
ou are evaluating a binary classifier. There are 50 positive outcomes in the test data, and 100 observations. Using a 50% threshold, the classifier predicts 40 positive outcomes, of which 10 are incorrect.What is the classifier’s Precision on the test sample?1 point25%60%75%80%
What does the ROC curve help determine in model evaluation?1 pointThe relative misclassification cost of the modelThe true-positive rate and false-positive rate for different criteria The optimal model based on diagnostic measuresThe model's statistical significanc
A model produces a perfect precision score of 1.0 for a positive class and sensitivity of 0.75. This means, the model may have mislabeled some positive instances as negative class. How can one judge the efficacy of the model in this scenario?Select an option Clear ResponseThrough F-measureThrough AUROCThrough accuracyThrough Specificity
Upgrade your grade with Knowee
Get personalized homework help. Review tough concepts in more detail, or go deeper into your topic by exploring other relevant questions.