nhorozov.xyz/blog/nemi26

NEMI was really cool

I flew back from Europe on August 12th (alone, for the first time :) ) so I used my jetlag to my advantage to get to Boston. I woke up at midnight to catch a 3am bus to Boston and I met a lot of really cool people there. I only met one other person who works on mech interp before this, and it was really great there.

I half wanted to go to NEMI to meet more people that do this, and half to see if I could do it.

There was one project I was really interested in, from the first student speaker, in which they built a method that finds directions in the loss landscape where the curvature is minimal, and moves the neural net in one of those directions (along with training the new network to have similar behavior as the inital one). These two steps are then repeated a lot. While I was watching the presentation I was thinking that the new neural networks would probably be harder to compress their weights that the inital ones, in the sense that their internal structures would be more complex. I later asked the presenter about how the rank of the matrices in the neural networks changes and she said for the RNN it increased, which made me really happy because that intution about compressablity was right :)

I wonder if this happens for other neural nets when you apply this method, that the rank increases.