products of deep networks, SKA

I spent the day (which is a Monday, but I am blogging it on a Sunday, because of international date-line insanity) with Marcus Frean (University of Victoria, Wellington) and Brewer (who came down to Wellington from Auckland). We spent the day arguing about what we should try to accomplish this week. When Frean and I last spoke, it was about probabilistic models for astronomical imaging, where he was trying to find subtle sources (like low surface-brightness galaxies) in radio imaging from the SKA pathfinders. Today we spent much of the day arguing about things we might try to accomplish this week, which ranged from tiny tweaks of things we have already done, or running existing software on new data, all the way to building a probabilistic generative model of all astronomical data ever (my usual plan!).

In the afternoon, we talked about deep networks and why they are powerful and yet tractable to train. I still don't fully understand, but I am closer. But then we discussed Frean's observation that most data sets are generated by multiple causes, so you really want not just a deep network, but a graphical model that has two (or more) independent deep networks pointing at the data. That is, we want products of deep networks, if we really want to represent the structure of the data in the world. We stared at that for a while; it appears to maybe require the probabilistic generalization of deep networks (about which Frean has no fear). He has some example code that operates the product of deep belief nets on toy problems!

At lunch, we were joined by Melanie Johnston-Hollitt (Victoria), who is deeply involved in New Zealand's (proposed, I think) buy-in to the SKA project. We discussed radio astronomy data analysis and the management and operation of global astronomical projects, including Sloan and SKA. We discussed New Zealand's role in the SKA; I am very pleased to see New Zealand in the SKA; this could be huge for New Zealand and for the whole astronomical community.