SDSS-V review, day 2

Today was day two of the SDSS-V Multi-object spectroscopy review. We heard about the spectrographs (APOGEE and BOSS), the full software stack, observatory staffing, and had an extremely good discussion of project management and systems engineering. On this latter point, we discussed the issue that scientists in academic collaborations tend to see the burdens of documenting requirements and interfaces as interfering with their work. Project management sees these things as helping get the work done, and on time and on budget. We discussed some of the ways we might get more of the project—and more of the scientific community—to see the systems-engineering point of view.

The panel spent much of the day working on our report and giving feedback to the team. I am so honored to be a (peripheral) part of this project. It is an incredible set of sub-projects and sub-systems being put together by a dream team of excellent people. And the excellence of the people cuts across all levels of seniority and all backgrounds. My day ended with conversations about how we can word our toughest recommendations so that they will constructively help the project.

One theme of the day is education: We are educators, working on a big project. Part of what we are doing is helping our people to learn, and helping the whole community to learn. And that learning is not just about astronomy. It is about hardware, engineering, documentation, management, and (gasp) project reviewing. That's an interesting lens through which to see all this stuff. I love my job!

chemical correlates of planet system architecture

At Stars group meeting, Jo Bovy (Toronto) demonstrated to us that the red-giant branch in Gaia DR1 TGAS is populated about how you would expect from a simple star-formation history and stellar evolution tracks. This was surprising to me: The red clump is extremely prominent. This project involved building an approximate selection function for TGAS, which he has done, and released open-source!

John Brewer (Yale) showed relationships he has found between planet-system architectures and stellar chemical abundances. He cleverly designed complete samples of different kinds of planetary systems to make comparisons on a reasonable basis. He doesn't have a causal explanation or causal inference of what he is finding. But there are some very strong covariances of chemical-abundance ratios with system architectures. This makes me more excited than ever to come up with some kind of general description or parameterization of a bound few-body system that is good for inference.

I spent the afternoon at CIS 303, a middle school in the Bronx, as part of their Career and College Day. This is an opportunity for middle schoolers to discuss with people with a huge range of careers and backgrounds what they do and how they got there. So much fun. I also ran into Michael Blanton (NYU) at the event!

#hackAAS at #aas229

Today was the (fifth, maybe?) AAS Hack Day; it was also the fifth day of #aas229. As always, I had a great time and great things happened. I won't use this post to list everything from the wrap-up session, but here are some personal, biased highlights:

Inclusive astronomy database

Hlozek, Gidders, Bridge, and Law worked together to create a database and web front-end for resources that astronomers can read (or use) about inclusion and astronomy, inspired in part by things said earlier at #aas229 about race and astronomy. Their system is just a prototype, but it has a few things in it and it is designed to help you find resources but also add resources.

Policy letter help tool

Brett Morris led a hack that created a web interface into which you can input a letter you would like to write to your representative about an issue. It searches for words that are bad to use in policy discussions and asks you to change them, and also gives you the names and addresses of the people to whom you should send it! It was just a prototype, because it turns out there is no way right now to automatically obtain representative names and contact information. That was a frustrating finding about the state of #opengov.

Budget planetarium how-to

Ellie Schwab and a substantial crew got together a budget and resources for building a low-buck but fully functional planetarium. One component was WWT, which is now open source.

Differential equations

Horvat and Galvez worked on solving differential equations using basis functions, to learn (and re-learn) methods that might be applicable to new kinds of models of stars. They built some notebooks that demonstrate that you can easily solve differential equations very accurately with basis functions, but that if you choose a bad basis, you get bad answers!

K2 and the sky

Stephanie Douglas made an interface to the K2 data that show a postage stamp from the data, the light curve, and then aligned (overlaid, even) imaging from other imaging surveys. This involved figuring out some stuff about K2's world coordinate systems, and making it work for the world.

Poster clothing

Once again, the sewing machines were out! I actually own one of these now, just for hack day. Pagnotta led a very successful sewing and knitting crew. Six of the team used a sewing machine for the first time today! In case you are still stuck in 2013: The material for sewing is the posters, which all the cool kids have printed on fabric, not paper these days!

Meta-hack

Erik Tollerud built some tools for the long-term storage and archiving of #hackAAS hacks. These leverage GitHub under the hood.

There were many other hacks, including people learning how to use testing and integration tools, people learning to use the ADS API, people learning how to use version control and GitHub, testing of different kinds of photometry, and visualization of various kinds of data. It was a great day, and I can't wait for next year.

Huge thanks to our corporate sponsor, Northrop Grumman, and my co-organizers Kelle Cruz, Meg Schwamb, and Abigail Stevens. NG provided great food, and Schwamb did a great job helping everyone in the room understand the (constructive, open, friendly, fun) point of the day.

#dsesummit, day 1

I'm at the Moore-Sloan Data Science Environments annual summit. Much of what we have been doing doesn't exactly count as research, by my (constantly weakening) standards. However, there was an absolutely great and wide-ranging discussion of Hack Weeks and Sprints and their role in education and scientific investigation. This led to a group of us committing to start a paper on the subject (not a white paper, but a paper). The just-started draft is here, and we accept pull requests.

There were some great lightning talks at dinner time. My personal favorite was Kellie Ottoboni (Berkeley) talking about the finiteness of the state space of random number generators. She (with Stark and Rivest) is looking at the possibility that there are random number generators possible with an infinite state space, capitalizing on the ideas around cryptographic hash functions. She sowed some (deserved) fear about using a 32-bit random-number generator in combinatoric contexts. Since our own emcee makes combinatoric choices, this could conceivably be relevant to our master branch!

#AAS227, day 4; AAS Hack Day

Today was the fourth annual AAS Hack Day (#hackaas) at #AAS227, organized by Kelle Cruz (CUNY), Meg Schwamb (Taiwan), and myself, and sponsored by the LSST Corporation and Northrop Grumman. We had a huge crowd: About fifty people and the staff had to bring in extra tables, chairs, and power strips. The hacks varied enormously in scope and category; here are just a few that stood out:

AAS meeting conflicts

Adrian Price-Whelan (and a bit Scott Idem and me) used some vector-of-words methods from previous AAS Hack Days to look at schedule issues in the AAS 227 program. He found pairs of oral sessions that were scheduled in conflict that contain talks with abstracts that are close in word space. The idea was to predict which sessions led to the largest number of complaints to the AAS about scheduling, and also provide prototypes of tools that might be used to make scheduling better in the future.

gender and questions in AAS oral sessions

Mehmet Alpaslan and a team including Hack-Day veteran Jim Davenport looked at new data on oral session question-askers and speakers and chairs, finding (as we learned at earlier meetings) that men ask more questions than women, but also finding that the gender of the speaker seems to be correlated with the gender of the question-asker. The data are barely understood at present, being only days old.

crowd-sourcing the old literature reference graph

In some twitter activity prior to the meeting, we discovered that old papers have poor citation and reference information, because the references were often in footnotes, formatted inconsistently, and OCR-ed badly. Brooke Simmons taught the AAS Hack Day participants how to build prototype Zooniverse projects for crowd-sourcing, and Brendan Wells used that knowledge to build a project to solve this old-reference problem. Love that collaboration, which was un-imagined prior to the Hack Day!

glassdome: glassdoor for astronomy

Ellie Schwab and friends started to build a site where people of all different ranks and seniorities could openly or anonymously review their home institutions, and comment on salary and other often-private things. Originally the project started as anonymous, but evolved to more encouraging of open and transparent reviewing as the day went on.

finding asteroids with Kepler

Geert Barentsen arrived with the retrospectively obvious point that the Kepler satellite is awesome for finding asteroids: It spends (in its K2 mode) half of its time looking inside the Earth's orbit, so it is great for finding Earth-crossing and inner asteroids. It also has great cadence and sensitivity. He assembled a great team and started to look. Science! Also on the science with Kepler tip, Jennifer Cash and Lucianne Walkowicz started work extracting photometry from full-field images.

death to Jet

Timothy Pickering propagated the new matplotlib non-Jet colormaps to plotly.js. This is God's work, as it permits web-plotting gurus to benefit from the latest research in visual perception of continuous data. In case you haven't been paying attention, Hack Days are a great time for people to bond over their hatred of the Jet colormap, but Pickering also reminded us of the research that shows that it leads to misconceptions about the data, fails in black-and-white printing, and is bad for people with vision impairments.

exoplanetary systems in WWT

David Weigel, after reminding us that World-Wide Telescope has gone open source and is now a project of the AAS, showed us how he put a known exoplanet system into the software. The plan is to get them all in there and then make possible tours and activities around exoplanet discovery and science.

fabric poster upcycling

Ashley Pagnotta and company brought a sewing machine to #hackaas. It turns out that it makes sense these days to print your poster on fabric not paper! This is because fabric printing is now very cheap, and you can pack a fabric poster trivially in your luggage. Check it out. But Pagnotta and colleagues brought patterns and skills and turned posters into infotaining clothing. Insane.

More (crowd-sourced but incomplete) notes are available here. Thanks to our sponsors and everyone who came, and see you next year!

AAAC, day 1

Today was the first day of the Astronomy and Astrophysics Advisory Committee meeting. The AAAC oversees the points of mutual interest and overlap between NSF, NASA, and DOE. For me the highlight of the meeting was consideration of the National Academy of Sciences “OIR Report”. It explicitly calls out the community and the funding agencies to make sure we are properly preparing students to work in data analysis, observing, and instrumentation.

These things were all standard in the astronomy of past ages. Then, instrumentation became more professionalized as instruments got more complicated (and more expensive), and fewer students built instruments. Then observing started to become professionalized with big surveys like SDSS and the like. And data analysis is headed the same way, as our precision goals get more challenging and our data sets get larger. What to do about all this is unclear. However it is abundantly clear that if we start to lose track of where our data come from or what has happened to them in processing, we are doomed.

#DSEsummit, day 2

In the morning, Katy Huff (UCB) gave an energizing talk about The Hacker Within, which is a program to have peers teach peers about their data-science (or programming or engineering) skills to improve everyone's scientific capabilities. The model is completely ground-up and self-organized, and she is trying to make it easy for other institutions to “get infected” by the virus. She had some case studies and insights about the conditions under which a self-organized peer-educational activity can be born and flourish. UW and NYU are now both going to launch something; I was very much reminded of #AstroHackNY, which is currently dormant.

Karthik Ram (UCB) talked about a really deep project on reproducibility: They have interviewed about a dozen scientists in great detail about their “full stack” workflow, from raw data to scientific results, identifying how reproducibility and openness is or could be introduced and maintained. But the coolest thing is that they are writing up the case studies in a book. This will be a great read; both a comparative look at different disciplines, but also a snapshot of science in 2015 and a gift to people thinking about making their stack open and reproducible.

I had a great conversation with Stefan Karpinski (NYU) and Fernando Perez (UCB) about file formats (of all things). They want to destroy CSV once and for all (or not, if that doesn't turn out to be a good idea). Karpinski explained to me the magic of UTF8 encoding for text. My god is it awesome. Perez asked me to comment on the new STScI-supported ASDF format to replace FITS, and compare to HDF5. I am torn. I think ASDF might be slightly better suited to astronomers than HDF5, but HDF5 is a standard for a very wide community, who maintain and support it. This might be a case of the better is the enemy of the good (a phrase I learned from my mentor Gerry Neugebauer, who died this year). Must do more analysis and thinking.

In the afternoon, in the unconference, I participated in a discussion of imaging and image processing as a cross-cutting data-science methodology and toolkit. Lei Tian (UCB) described forward-modeling for super-resolution microscopy, and mentioned a whole bunch of astronomy-like issues, such as spatially variable point-spread function, image priors, and the likelihood function. It is very clear that we have to get the microscopists and astronomers into the same room for a couple days; I am certain we have things to learn from one another. If you are reading this and would be interested, drop me a line.

#AstroHackWeek 2015, day 5

In the morning, tutorials were again from Brewer and Marshall, this time on MCMC sampling. Baron got our wrap-up presentation document ready while I started writing a title, abstract, and outline for a possible first paper on what we are doing. This has been an amazingly productive week for me; Baron and I both learned a huge amount but also accomplished a huge amount. Now, to sustain it.

The hack-week wrap-up session blew my mind. Rather than summarize what I liked most, I will just point you to the hackpad where there are short descriptions and links out to notebooks and figures. Congratulations to everyone who participated, and to Huppenkothen, who did everything, and absolutely rocked it.