Food additives and medications (antibiotics) are biggies. Pollution has been reduced to about half what it was when I was a kid, so I consider that a non issue. All this sterilizing of the environment, in my opinion, is the biggest culprit for our weakened immune systems. In order to build a resistance to germs, we need to be exposed to said germs. Killing them to keep from getting sick creates a viscious circle. We were healthier when we were not so sterile.
I ate dirt as a young girl. Most of us who played outside, made mud pies and had sand boxes did. We didn't have our moms chasing after us with Clorox wipes and sterilizing everything before we played with it. We washed dishes with the same rag for a week before changing to a clean one. We ran around barefoot. We shared ice cream cones.
We would spread germs around like parade candy and people didn't get sick. Because? Well, because we had a resistance built up to the germs and our immune systems were prepared for them.