Fishermen have warned the Environment Secretary, Michael Gove, that Brexit is the last chance to save their floundering industry.