I live in San Jose and have been here for about 10 years. How long is your trip for. If its just a few days, the things to do in SF/Berkeley will keep you busy. Obviously do the obvious things like ride a cable car in SF, go to Lombardi Street (crooked street), Fisherman's Wharf, Union Square, and China Town.
If you are in San Jose, check out a Sharks game, they might be playing when you come here and tickets arent as bad as Cowtown from what I remember, although I havent been to a game in Calgary in a really long time. Even if you dont have a car, you can take the Caltrain right from SF to SJ (and the station in SJ ends up being right across the street from HP Pavilion where the Sharks play, its called San Jose Diridon). And then on the way back, you can take Caltrain back to SF and get off right by where the Giants play!
I cant remember if its baseball season (not really a fan) but definitely go to a Giants game as the ballpark is gorgeous, right in San Francsico, and right off the water. Beautiful view as some have mentioned. It gets chilly though so bring a sweater.
But as some people mentioned, if you have a few extra days, check out Big Sur (I did a half marathon there last year and it was AWESOME). I lived in Monterey for a year when I first got married and that area has some of the most beautiful places on earth...17 Mile drive is gorgeous, Carmel beach is nice too. But its a bit far if you're only in SF for a few days.
What else are your interests? Are you into nightlife - clubs or bars?