Coalition member goes AI

My coalition partner said he turned off his recruitment centers and was quitting. The problem is that he didn't move all his armies out of the way. I have moved into his unoccupied provinces.

Should I kick him out of the coalition now or wait until he goes AI?

What is the best way to handle this.