Check model stability and adversarial vulnerability

[DJCordhose]

Main question: does slightly perturbing the input data yield a drastically different risk or a different risk group. If so

• there is an additional attack vector because people could learn decision boundaries and by slightly tweaking features that do not require exact entry (like estimated miles per year) get into a better category, thus being able to hack the system
• high local variation hints towards undetected overfitting
• high local variation makes it likely that retraining with new data will yield a completely new model which also requires new interpretation etc.

Links:

• https://en.wikipedia.org/wiki/Metamorphic_testing