TV chef Andi Oliver has said culture will only improve when “we stop all the fake outrage and actually implement change” amid the Gregg Wallace furore.