I don't think it counts as 'one that got away' if you never had a chance in the first place. When people say that, they mean someone who was REALLY interested them but neither person ever made a move and they both went on to different things.
Reference 'Sk8er Boi' by Avril Lavigne.