I think the main issue I have, and likely many others too, is how strongly it is phrased. If he thinks he'll die in the next 5 or 10 years... fine, I guess? But that's unlikely, and with how things have shifted just in the past 25 years, making a statement like this seems arrogant.
Is that overanalysing a one line answer to a question? Probably, but that's what a politician gets and the effect of modern media.
Not to mention how the population and especially Labour supporters have turned pro-EU so he'll likely alienate that part of his base. Strong stances are seen as better, but I really feel sometimes he should take a softer approach.