Right now I'm supporting Herman Cain for President in 2012 and yet people still think I'm racist...?

Why is it that my liberal and republican friends either call me racist or tell me not to openly voice my view that President Obama is horrible because people will think I'm racist? I'm liking Herman Cain for the Presidency so far so why am I still considered racist?