Dimensionality reduction (the basic being PCA) is very sensitive to outliers: A single bad pixel can dominate most objectives and thus create a spurious dimension. One of the best and most classic solutions to this is the robust PCA method, which is presented in a (very long) paper with impressive math and beautiful results. Yesterday Hans-Walter Rix (MPIA) and I coded it up and applied it to ESA Gaia RVS spectra, with extensive (and impressive) help from Claude. It looks very promising, especially in capturing oddities in hot stars. Today I worked out that there should be something similar that takes into account data weights (inverses of squared uncertainties), and I wrote down the algorithm (on paper). We'll see.
No comments:
Post a Comment