The airport rocket to Kipling station is definitely a popular choice. A cab is about $40, and will take about 25 minutes.
So you start at Union Station (between Front and Bay, and Front and York)
About 5 blks East is St. Lawrence Market - an old institution here in TO. If this holds any interest, you could come walk around and engorge yourself on various meats, cheeses, etc. It opens at like 5am on Saturdays.
http://www.stlawrencemarket.com/shopping/south.html
Back at Yonge and Front is the HHOF. Easy.
About 7 large blks west of HHOF is the Rogers Centre. About a 5 minute walk. If you want a little variety, maybe walk down to the Lake (Queens Quay) and walk west along there, then you can walk back up (Rees St) for the game.
http://maps.google.com/maps/ms?ie=UT...148bafc9bf5b4d
jaywalk all you want.