in a nutshell, seconds per lap add up to minutes per race. it's a given that for most courses, a ss is going to lose time at various parts of a course compared to a geared bike - you don't need a very long descent or stretch of flat terrain to jump on a big gear for 5 to 10 seconds, or a very long "climb" to take advantage of a smaller gear. not to mention that it's extremely difficult for a ss'er to make the selection for the front group, or hang with a chasing group, without having a true sprint or the ability to cover attacks.
so, while the fastest guys and gals on a ss can minimize these disadvantages, losing maybe 15-20 seconds per lap in a 6 lap race is . . well, shit - 15-20 seconds is a lot time. all things being equal, 90-120 seconds (and that's being very generous to a ss'er) is not what I'd describe as near parity.
and, yeah - of course this will vary for a given rider and what field they race in. but for the majority of riders, ss is a competitive disadvantage. which is why there is an awesome ss category at lots of races these days!