Sports minister Nigel Huddleston says he wants to see football fans return to stadia 'as soon as it is safe to do so'.