Table 3 Quantitative impact of risk-based approach. Differences of expected risk \(\:\stackrel{\sim}{ER}\) when varying the risk ratio \(\:{c}_{FN}\) systematically between \(\:1.0={2}^{0}\) and \(\:16={2}^{4}\) (stepwise increment by factor 2) as well as \(\:{c}_{FN}=10.0\) as an extra point of evaluation. Due to symmetry reasons, the values for \(\:{c}_{FN}<1.0\) are equivalent to the inverse risk ratio \(\:\frac{1}{{c}_{FN}}\). The rightmost column shows the relative differences between \(\:\stackrel{\sim}{ER}\left({s}_{{c}_{FN}}\right)\), i.e. the value at the optimum position \(\:{s}_{{c}_{FN}}\) for the particular curve, and \(\:\stackrel{\sim}{ER}\left({s}_{1.0}\right),\) i.e. the value at the default threshold \(\:{s}_{1.0}\)
From: Risk-based evaluation of machine learning-based classification methods used for medical devices
Parameter settings of artificial model / risk ratio | Optimum threshold \(\:{\varvec{s}}_{{\varvec{c}}_{\varvec{F}\varvec{N}}}\) and corresponding \(\varvec{\:\stackrel{\sim}{{E}{R}}}\) value | Comparison of \(\varvec{\:\stackrel{\sim}{{E}{R}}}\) values: \(\:{\varvec{s}}_{{\varvec{c}}_{\varvec{F}\varvec{N}}}\) vs. default threshold \(\varvec{\:{\varvec{s}}_{1.0}}\) | |||
|---|---|---|---|---|---|
modified Gaussian \(\varvec{\:{{\sigma\:}}_{{F}{P}}\:/\:{{\sigma\:}}_{{F}{N}}}\) | risk ratio / weight \(\varvec{\:{c}_{FN}\:/\:{c}}\) | optimum threshold \(\varvec{\:{s}_{{c}_{FN}}}\) | expected risk value | relative difference \(\varvec{\:\frac{\stackrel{\sim}{ER}\left({s}_{1.0}\right)}{\stackrel{\sim}{ER}\left({s}_{{c}_{FN}}\right)}}\) | |
at \(\varvec{\:{s}_{{c}_{FN}}}\): \(\varvec{\:\stackrel{\sim}{ER}\left({s}_{{c}_{FN}}\right)}\) | at \(\varvec{\:{s}_{1.0}}\): \(\varvec{\:\stackrel{\sim}{ER}\left({s}_{1.0}\right)}\) | ||||
\(\:{{\sigma\:}}_{{F}{P}}=0.1,\) \(\:{{\sigma\:}}_{{F}{N}}=0.1\) | 1.0 (default) | 0.5 (default) | 0.08 | 0.08 | 1.0 |
2.0 | 0.46 | 0.11 | 0.12 | 1.07 | |
4.0 | 0.44 | 0.16 | 0.21 | 1.30 | |
8.0 | 0.40 | 0.21 | 0.37 | 1.77 | |
10.0 | 0.38 | 0.23 | 0.45 | 1.98 | |
16.0 | 0.36 | 0.27 | 0.70 | 2.58 | |
\(\:{{\sigma\:}}_{{F}{P}}=0.2,\) \(\:{{\sigma\:}}_{{F}{N}}=0.2\) | 1.0 (default) | 0.5 (default) | 0.29 | 0.29 | 1.0 |
2.0 | 0.44 | 0.40 | 0.43 | 1.08 | |
4.0 | 0.36 | 0.52 | 0.72 | 1.38 | |
8.0 | 0.3 | 0.65 | 1.29 | 1.97 | |
10.0 | 0.26 | 0.70 | 1.58 | 2.26 | |
16.0 | 0.22 | 0.78 | 2.44 | 3.12 | |
\(\:{{\sigma\:}}_{{F}{P}}=0.3,\) \(\:{{\sigma\:}}_{{F}{N}}=0.3\) | 1.0 (default) | 0.5 (default) | 0.43 | 0.43 | 1.0 |
2.0 | 0.4 | 0.59 | 0.65 | 1.10 | |
4.0 | 0.3 | 0.75 | 1.09 | 1.44 | |
8.0 | 0.18 | 0.89 | 1.96 | 2.20 | |
10.0 | 0.16 | 0.92 | 2.39 | 2.59 | |
16.0 | 0.08 | 0.98 | 3.69 | 3.78 | |
\(\:{{\sigma\:}}_{{F}{P}}=0.4,\) \(\:{{\sigma\:}}_{{F}{N}}=0.4\) | 1.0 (default) | 0.5 (default) | 0.54 | 0.54 | 1.0 |
2.0 | 0.36 | 0.72 | 0.80 | 1.11 | |
4.0 | 0.22 | 0.88 | 1.34 | 1.51 | |
8.0 | 0.08 | 0.98 | 2.41 | 2.45 | |
10.0 | 0.04 | 1.00 | 2.94 | 2.96 | |
16.0 | 0.00 | 1.00 | 4.55 | 4.55 | |