Andrew Pierce is all too aware of his “terrible stereotype”: “that I’m some hardline unsympathetic Brexiteer Thatcherite”.