Tony Blair has said Labour would 'annihilate' the Conservative Party were it to take an opposing stance on Brexit.