I think you just made the arguement for both sides, CI.
As a speeder; I only have to worry about the traffic in front of me- the direction my eyes are pointing. If I were to go too slow, then I have to worry about speeders coming up behind me. And because I know that I have above average driving skills, I know that I should be more worried about the people I can't see behind me, as opposed to me being worried about the people in front of me that I can see and use my above average skills to avoid.