adversarial attacks and model derivatives

I froze in my mother-in-law's car (NYC alternate-side parking FTW) while I spoke with Teresa Huang (JHU) and Soledad Villar (JHU) about our old project to find adversarial attacks against machine-learning methods used in astronomy. One of the big problems we face is that our methods require good derivatives of output with respect to input (or vice versa) for the methods we are studying. However, it is often hard to get these derivatives precisely. Even when a method has analytic Jacobian or derivative operators (like tensorflow and jax deliver), it isn't always exactly useful, because sometimes the methods are doing stochastic things like dropout and ensembles when they make predictions. Our conclusion was that maybe we need to be reimplementing all methods ourselves, maybe in straw-person forms. That's bad. But also good?