How long has there been only 4? Sometimes there is a day lag depending on when you got down to 4 players.
Other Possible explanations.
1-Somebody keeps logging in for a minute and logging back out keeping the timer going.
2-It has to get down to just 3 players to vote to end the round?
Lol, thanks for your reply. Weve been down to 4 for a least a week, probably more. It's a glitch, I've ran into before. We are down to 3 today 
Thank you game guys, we have the vote now 
So, did it need to get down to 3? I always thought that was the rule before retirement is allowed.
Almost forgot that this feature existed.