Model Evaluation (A Second Look) in Logistic Regression

In a previous session on model evaluation, you learnt about various model evaluation measures, such as accuracy, sensitivity, specificity, KS statistic, etc. Now, let's look at some more measures commonly used for model evaluation.

Note: At 02:29 our SME's says "%Goods in X-axis and %Bad in Y-axis", please note that it should be %Bad in X-axis(FPR) and %Good in Y-axis(TPR).

Following shows the ROC curve for 3 cases, i.e. the random model (orange line), the ideal model (grey line), and the real model(blue line), in our case, the telecom churn model.

Clearly, the perfect model is pretty much a right triangle, whereas the random model is a straight line. Basically, a model that rises steeply is a good model.

Another way of saying that is — it will have a higher area under the curve. So, the Gini coefficient, which is nothing but a fancy word and is given by -

$Gini=AreaUnderROCCurve$

will be high for a good model.