I drive an 05 Legacy GT and would definitely buy another one if/when my current ride dies or is written off by something discussed in the 'Grind My Gears' thread.
A few negatives that you may not notice right away:
Although the car is solid and runs great, you'll pay for it via the boxer engine. My opinion is that the tank is on the smallish size for its consumption and depending on what model you get; the frequent refilling of Premium fuel is a bit of a kick to the pants when you are used to driving lighter or more fuel efficient cars.
The OEM tires are garbage! I was lucky enough to be able to swap my OEM tires for far decent summers on the cheap and was able to find stock rims from an owner in Colorado for my winter rubber. I found it a bit difficult finding wheels that fit my Legacy for a decent price (granted it was 6 years ago).
Servicing - We only have two dealerships in Calgary and I've got my gripes with both of them. Although the vast majority of staff I've dealt with have been wonderful at both locations, there is one 'service' person at each location that I'd like to see removed. Apparently Subaru Specialists are great to deal with and I'll be taking my service there once my extended warranty runs out.
I should note however that I may have a heavy foot and maybe I can be impatient behind the wheel so my experiences will be different than yours. If you're into forums check out legacygt.com. ...It's another great community of like-minded enthusiasts!