You don't forget, but you do fall out of practice doing what are essentially competitive programming exercises.

When you're in a situation in front of a whiteboard with 15-45 minutes to craft some algorithm it makes a ton of difference whether or you've been practicing.