Bobby Fischer was an American chess grandmaster and the eleventh World Chess Champion. Recognized as a prodigy, he won multiple US Championships and held the world title from 1972-1975. Fischer made significant contributions to chess with his book 'My 60 Memorable Games' and the invention of Chess960.
This is based on winning 6 head-to-head matchups with other famous people, but also losing 16 head-to-head matchups.
Most recently Bobby Fischer lost a matchup to Robert Downey Jr. (#8) voted by someone from Canada.