Protesters gathered at Downing Street chanting "Stop Brexit!" as PM Boris Johnson was making a speech.