Late switch to Custom User Model always hurts?

Yes! I agree with Matt there — before finally deploying, and letting others loose on the code is the time if you can (and you want to).