In physics, your solution should convince a reasonable person. In math, you have to convince a person who's trying to make trouble. Ultimately, in physics, you're hoping to convince Nature. And I've found Nature to be pretty reasonable.

I went off to college planning to major in math or philosophy-- of course, both those ideas are really the same idea.