Open and honest communication. Just say that you're not interested in the game.
If the host gets passive-aggressive you could try to talk civilised about that. In case they don't show the necessary understanding or are difficult to talk to, they might not be the kind of acquaintance you would like to keep.