What I'm trying to say is that just because I am insecure doesn't mean that I'm wrong.
It's not a unique position, but it's not universal either. Some people have insecurities that relate to other problems. Some people go out and think "I look hot," and mean it. They might doubt their social skills or intelligence or worth as a human being, but they know that people find them attractive. Some people doubt their appearance, but with far less reason. I think that my attractiveness to other people is below average, which by definition means that half the population is better at it than I am. So that half of the population either knows it, and feels less doubt, or doesn't know it, and their doubt is less rational. Either way, it's not the same thing.