That's a terrible framing of the situation. The opinion (rightly or wrongly) is that cheap labour from poorer countries sets the expectations for employment costs.
Point taken, partly: It's not just the wages (and the accommodation) that are shite. The work itself is monotonous and physical.
If labor costs increase that may lead to automation to taking the place of migrants. That would mean a low number high-paying jobs for the well-educated rather than a large number of high-paying jobs for the poorly-educated.
Migrants who stay for longer to some degree bring their own jobs and economy with them anyway — all those Polish delis the article alludes to have Polish shop-owners. Without the migrants, there'd be no need for Polish shops.
Brexit was a sledgehammer approach for people to say "no jobs should pay so low that you have to live a subsistence existence."
Just as much as Brexit was a viable approach to addressing NHS financing, I guess. The EU never stopped the UK from enacting sensible social or sensible healthcare policy. But I understand some people may have been duped.