# Preformance Measures

Classification Based Problems

A self explanatory table from Wikipedia called Confusion Matrix is as shown below.

Confusion Matrix

Confusion Matrix	Really Positive	Really Negative
Predicted to be positive (P)	TP	FP
Predicted to be negative (N)	FN	TN

If you consider classification of images into Cat and Not a cat classes then the preformance of a given model can be measured as follows:

Recall a.k.a True Positive Rate

Of the all the Cat images, how many were we able to sucessfully able to detect.

$\text{TPR or Recall} = \frac{TP}{TP + FN}$
False Positive Rate

Of all the Non-cat images, how many of them were classified wrong.

$FPR = \frac{FP}{FP + TN}$

ROC – Receiver Operational Characteristics

Plot of TPR vs FPR

Why is it important?

Example 1: Consider a following senario: A radar is receiving some signal and our job is to classify the signal into Enemy Plane and Noise classes. For that, we have to set appropriate thresholds. 1. We need to set threshold such that the TPs are high and we certainly don’t want False Negatives (Classifying enemy plane as Noise). Hence we are more concerned about False Negatives and we must ensure lower FNs. 2. And we can compromise on FPs. Hence, even if the FPR is high, it is Okay!

Example 2: Consider another senario where you are deteting tumor cells. 1. In this case, even if the model misses to detect the tumor cell correctly, it Okay! Doctors can do additional tests to detect the tumors. So, FNs are not that critical. 2. But, if the model says that a patient is ill when he isn’t, that is serious. So, FP should be as low as possible and hence FPR becomes the important factor.

Precision > Of the images that are predicted to contain cat, how many of them really contain cat?

$Precision = \frac{TP}{TP + FP}$

F1 Score

Hormonic mean of Recall and Precision.

$F1 = \frac{2}{\frac{1}{Recall} + \frac{1}{Precision}}$

Regression Based Problems

R-squared value

The worst prediction that you can do is to just predict the average of all the expected output ( $\bar{y} = \frac{1}{m}\sum y$ ) for any input. The error due to this model is called $SS_{tot}$ .