I second fisherman's wharf, Eaglewing whale watching is there and my wife and I had a blast with them and at the wharf. Eaglewing tours also has a no whale sighting, no pay policy so if all you see is the seals at Race Rocks Lighthouse the trip is free.
Also the Salish Sea Aquarium is fantastic, it is all local sea life and was not overly crowded when we went. I liked it more than the Vancouver aquarium because of that.
http://www.oceandiscovery.ca/
And there is a lot of artists there to see and buy from, one I wish I had bought from was a gentleman who did some beautiful work with seaweed that was like leaf or flower pressings. The seaweed looked like it was a painting because of how he poses it.
We went last year at the end of May.