Game not allowing to rejoin

I had a game I was playing but now when I try to join it tries to load but then goes back to the game menu. How can I fix this to join the game ?