I met early (by videocon, of course) with Teresa Huang (NYU) and Soledad Villar (NYU) to talk about our projects to develop adversarial attacks against regressions of discriminative and generative forms. We ended up talking a bit about information theory. I gave my minimal description of Fisher Information. I was recalling that I was taught some of that back in my PhD, but I forgot it all and re-learned it by using it in real data analyses. I feel like it would be a good subject for an arXiv-only post.
The question to hand today was this: You are given a set of data x that contain information about some quantity y. For a training subset, you are also given labels y, which are noisy. That is, the labels you are given do not exactly match the true values of y. Which contains more information about the true labels? The labels you are given or the data? This is a question answerable (under exceedingly strong assumptions) within information theory.
No comments:
Post a Comment