2020-03-31

latent variables in generative models

I worked a bit on my project to compare generative and discriminative models. As my loyal reader knows, I love generative models and am suspicious of discriminative models, and I'm trying to understand whether my prejudices are justified in any way. I have been thinking about this in terms of adversarial attacks, and also in terms of information theory. I keep finding (to my delight) that discriminative models are worse on both counts. However, I have been unfair, because I have been fitting generative models that are precisely matched to how the data are themselves generated. I worked out a more general setting today, where the generative model will be appropriately and realistically wrong; will it still be more robust against attacks? The main idea is that in the real world, there are latent variables you can't know, even for your training data. And you not only don't know these latents, you don't even know how many there are, or how nonlinear their effects.

No comments:

Post a Comment