I don't know most of the fandoms either, but I read everything and treat the unknown fandoms as original fiction. My votes are based on the quality of writing and how much the story engages me, so I don't automatically vote for fics in familiar fandoms. Most of the unfamiliar fandoms are things I'm never going to watch/read anyway, and fanfiction by its very nature means that writers don't have to stick to canon anyway, so I set any concerns about the source material aside and vote based on each story's merits as a piece of fiction. It seems the fairest wsy. =)