So, is there really any reason for him to be mad?

I'm not convinced that he really wants to leave. I think his ego is getting in the way for him to say something. Don't kick him out nor encourage him to leave. See if he wants to go to marriage counseling to pick up some tools for keeping it all together. Maybe something good can come out of it. After all, it did for 16 years. Good luck.