I have never noticed any such correlation among the people I know. I have known a few racists who didn't seem to care about fitness. I haven't known any racists who fretted over fitness. I have known lots of non-racists who care about fitness, and lots of non-racists who don't care about fitness. I just haven't seen any correlation here. I realize that my empirical results are standard "small number statistics", but if the phenomenon were as strong as you suggest, I'd expect to have observed it.
I doubt that your hypothesis is susceptible to empirical confirmation or rejection, but not for internal reasons; it's just that such a survey would be a huge operation.
On a theoretical level, I don't think that the historical factors you cite have much operational significance today; not many people are even aware of the historical factors you cite. For most people, concern over fitness is what I call a "one-dimensional attribute"--a strictly internal concern based on the desire to be attractive, a desire to be healthy, or the fear of mortality. The two-dimensional aspect of concern over fitness is much weaker--yes, I do feel some contempt for an obese person, but that's a fleeting thought.
Perhaps the problem here is the rarity of genuine, red-blooded racists. There are far more people who are unconcerned or callous about racial issues.