This may sound strange.... but if someone else tells you that.... I think you should respond with "Oh really?" and ask why the person thinks so (sarcasm, of course).
I'm not saying anything is wrong with you, but if you don't already know what people are thinking about you, it may be a good idea to find out. (I know once someone spread bad rumors about me. That's why I'm saying it. Also, sometimes, I look quite unfriendly, which is the opposite of how I really am.)
Unfortunately, I get the same thing. Personally, I think it's kinda funny, but I'm a gal, though. Everyone thinks I'm gay, but that's okay because....
THEY ARE NOT WORTH MY TIME. LATER FOR THOSE PEOPLE. I'll stay by myself before I entertain them. (Hmph! ~waves a finger~ I'm not going to say which one.

I know how you feel, though.... I feel the sex-relationship thing can add to depression (like when the person doesn't treat you properly) as well as subtract from it (relieve tension, etc., if the situation is a good one)....
Anyway, it may be hard to find someone, but are you looking? Where are you looking? How are you looking?
Oftentimes, I feel I can't connect with people where I live.... Can you try another state/county/spot a bit away from where you live? Or meet someone through what you love to do?
I'll be honest and say that I was happier after having sex/making love with someone I liked. Don't give up hope! Be a little bold, if you are not usually so. ***Just be careful....but go for it!!*** <---(That's my motto.)
I hope you find someone soon!