faster iterations, but a whole lot more of them

On my last day at MPIA—what a great summer it has been—Tsalmantza and I raced an iterated gradient descent against a block-diagonal method for solving the matrix factorization problem we have been working on (I have given it many names, but I now realize that Roweis would have called it matrix factorization). We were both sure that gradient descent would crush because in our (not brilliant) R implementation, the gradient descent steps took hundreds of times less time to execute than the block-diagonal steps. But it turned out that gradient descent takes more than hundreds of times more iterations! So the block-diagonal method wins. We probably could beat both with conjugate gradient, but that will wait for another day. I have packing to do!