If you're around the wharf anyway, go get breakfast at the Boudain bakery. One of my favorites when I was down there was the Magnolia Brew Pub on Haight.
http://www.magnoliapub.com/index.html
I didn't do any really fancy dinners when I was down there so I can't recommend any upscale dining, though.