Question

A classifier of fake banknotes is being tested to see how reliable it can detect real banknotes from fake ones. Three confusion matrices representing 3 Days of testing was made 

Day 1

n = 100

Predicted No Predicted Yes

Actual No - 15 35

Actual Yes - 8 42

Day 2

n = 100

Predicted No Predicted Yes

Actual No - 22 28

Actual Yes - 2 48

Day 3

n = 100

Predicted No Predicted Yes

Actual No - 43 7

Actual Yes - 16 34

1)Which day has the worst CSI?

A) Day 1

B) Day 2

C) Day 3

D) Insufficient Information. Cannot be determined

2)The overall F-score of the test is around? 4 decimals places

3) If the target of the study is to have a total CSI above 60% and an F-Score of above 70%, does the classifier pass? True or False

Answer 1

1)

If CSI is sensitivity, then:

$CSI=\frac{TP}{TP+FN}$

where TP is true positive, FN is false negative.

$CSI_1=\frac{42}{42+8}=0.84$

$CSI_2=\frac{48}{48+2}=0.96$

$CSI_3=\frac{34}{34+16}=0.68$

Answer: A) Day 3 has the worst CSI.

2)

F-score:

$F=\frac{2TP}{2TP+FP+FN}$

where FP is false positive

$F=\frac{2(42+48+34)}{2(42+48+34)+(35+28+7)+(8+2+16)}=0.72$

3)

True. The classifier passes, because F-score of the test is more than the target of the study.