What's an adversarial attack against a regression?

I had a very brief but useful conversation today with Soledad Villar (NYU) about the strategy and meaning of adversarial attacks against regression methods. We have been working on this all semester, but I am still thinking about the fundamentals. One thing I am confident about, even in the trivial machine-learning methods I have used in astronomy, is that there will be successful single-pixel attacks against standard regressions that we use. That is, you will find that the ML method is very sensitive to particular pixels! But this is a conjecture. We need to make a very clear definition of what constitutes a successful attack against a regression. In the case of classification, it seems like the definition is “The authors of the method are embarrassed”. But that doesn't seem like a good definition! Aren't we scientists? And open scientists, at that.