People are disposed to see themselves as rationally evaluating their political choices, whether it be on dimensions that commonly are loosely described as liberal or conservative. Actually, that dimension, more continuous than dichotomous, is further complicated by there being at least three factors: (a) the social liberal/conservative factor, (b) the economic liberal/conservative factor, and (c) the degree of government intervention factor. And, obviously, there can be very practical or philosophical reasons a person may cite for his choice.
But there's another, somewhat less admirable factor, that can also play a point. That's being influenced in your voting by not wanting to be affiliated even in that way with people you hold in contempt.
Let's use the example of the Presidential election of 1884. In that year, Grover Cleveland defeated James G. Blaine because Cleveland won New York because of what a Dr, Samuel B. Burchard said in a speech:
We are Republicans, and don't propose to leave our party and identify ourselves with the party whose antecedents have been rum, Romanism, and rebellion. We are loyal to our flag.
That remark antagonized opponents of Prohibition and Catholics, both major sources of votes.
In short, Blaine was screwed by one of his supporters' unfortunate remarks.
And this can happen at other times. I call it the Asswipe Factor in Politics: sometimes a candidate's supporters can be a burden. The candidate may be perfectly reasonable, but he's surrounded by a bodyguard of idiots.
After I voted for President in 2008, I had a most depressing thought: I voted for the same guy that Bill Maher, Jon Stewart, and Paul Krugman supported. It spoiled for me any pleasure in my having voted that day. (In compensation, if I had voted for the other candidate, I would be sobered by the thought that I voted for Rush Limbaugh's candidate!)