What to do about a horse being head shy and trying to eat non edible thinks?

Buster is 9 and he is head shy (we bought him that way) and he tries to eat non edible things (e. flashlight,bucket, hair, lead rope, etc) well then the problem is when he tries to eat something hes not supposed to and u try to take it from him he jerks his head up (he is a dun quarter horse he probably 16 hands high) now also taking his bridle off is hard also cause he jerks his head up (he as broken maybe 3 or 4 bridles) now sometimes when he jerks his head up his head up the bridle falls off (which usually means he broke it) now once it didn't fall off and he almost pulled my shoulder out of socket so what can i do about his head shyness and trying to eat non edible things