2014-05-12

causality, web scraping for science, visualization

A great day! Avoiding proposal-writing, I guess: I met with Jennifer Hill (NYU Steinhardt) to discuss ways we might bring causal inference to astronomy. The idea is not to improve what is being done or concluded in astronomical studies, but rather to clarify the implicit assumptions. We tentatively decided to look at galaxy evolution, because it is a sub-field in which the models are currently very data-driven and aren't backed by strong theoretical ideas. It all might fail, of course, because it is hard to think of things like galaxy environments as "treatments" and things like star-formation rates as "outcomes". But the problems do map onto each other, I think.

At lunch, Raphael Flauger (NYU) gave a beautiful talk on foreground uncertainties related to the BICEP2 results. He built his foreground models as did the BICEP2 team by scraping data out of Keynote (tm) presentations posted on the web! I have to say that again: The Planck team showed some maps of foregrounds in some Keynote presentations and posted them on the web. Flauger (and also the BICEP2 team before him) grabbed those presentations, scraped them for the all-sky maps, calibrated them using the scale bars, and worked from there. The coolest thing is that Flauger also simulated this whole process to account in his analysis for the digitization (scraping?) noise. Awesome! He concludes that the significance of the BICEP2 results is much lower than stated in the paper, which makes him (and many others) sad: He has been working on inflation models that produce large signals.

Late in the day, Enrico Bertini (NYU Poly) rocked the Data Science Showcase series with a nice talk on visualization. Among many other things, he showed that there are known rankings of the value of different kinds of visual cues (bar lengths, dot positions, color, and so on) for displaying different kinds of quantities (real value, rank, binary). Afterwards, there were some conversations about what we should do about scientists who are blind. That's an important question we need to address when we speak about the importance of visualization!

2 comments:

  1. Flauger's slides are up in a modified form here http://www.pctp.princeton.edu/pctp/SpecialEventSimplicity2014/SpecialEventSimplicity2014.html along with video

    ReplyDelete
  2. Automate Everything with web bots! This actually made my life easier... https://simplestipsandtricks.blogspot.com/2018/10/the-power-of-headless-chrome-and.html
    Make your Own Web Crawler - Web bots

    ReplyDelete