2006-03-06

MW satellites, CVS repository destroyed

Willman gave a great group-meeting presentation about the status of the ultra-faint companions to the Milky Way discovered by herself and others as overdensities in the SDSS stellar catalogs. There is so much more to be done, and our current estimate is that there will be dozens of new satellites discovered when SDSS-like (and better) surveys cover the whole sky. Awesome!

In other news, the Princeton CVS repository (with all the Princeton-based SDSS data analysis code, including all the spectroscopic reduction code, and including several huge code bases to which I am a large contributor and upon which I rely heavily) got destroyed today in a RAID failure. Did you hear me? All the Princeton SDSS data analysis code! All! Let's hear it for data integrity: A couple of times per day I rsync the Princeton CVS repository to NYU, and the NYU copy saved the day. The only stupid thing is that I didn't charge for this service! I think I need an agent.

BTW and IMHO, off-site backup is the only way to go, and I strongly suggest to all you large-data and large-code-base people out there that you arrange mirroring trades (as in I'll mirror your dataset if you mirror mine) with your collaborators at distant institutions. A few minutes of finding disk space and setting cron tabs can save you a lot of future hell.

5 comments:

  1. And this is why I had my boss by an external hard drive, so I could mail it off to the PI for my data's institution. They copied the archive over to it, mailed it back, and I spent 25 hours burning to DVD ASAP.

    Oh so worth it, both to have data locally and to have archives in different places on different media (different failure mechanisms).

    ReplyDelete
  2. Nothing can save us from future Hell.

    ReplyDelete
  3. Lets HOPE all the the LEPs and Tevatrons data to be there kept for reanalisis after the LHC starts to give us new physics.

    ReplyDelete
  4. Your headline makes it sound like the satellites have been destroyed along with the repository.

    ReplyDelete
  5. Mathew: Actually, some of the MW satellites have been destroyed (though I admit that the headline was ambiguous).

    ReplyDelete