There's my mistake. From the FoI Wiki, it was Cat recounting the story in her head, and then asking Ned about it:
"Another theory claims Jon's mother was
Ashara Dayne, his father being likely Eddard Stark or his older brother
Brandon Stark. Lady Dayne was said to be with both of them during the
Tourney at Harrenhal. According to Ser
Barristan Selmy, one of the Starks bedded her at the tournament and got her with child. It was also known by many that she was pregnant before she killed herself by jumping from the tower of
Starfall, also called the Palestone Sword, a tower on Starfall's cliff by the sea. Though Ser Barristan believes the ####### child of Lady Dayne was a stillborn girl, many still believe Ashara Dayne to be the mother of Jon Snow. Among those who believe so are Queen
Cersei Lannister and Lady
Catelyn Stark. Eddard refuses to answer Catelyn when she asks if Ashara Dayne was Jon's mother."
Told you my memory was a bit fuzzy.
