My wife and I got married in Jamaica, and it was awesome. However we didn't invite anyone!
We had a reception when we got back where everyone could come, which works out well.. everyone gets the benefit of the party, and doesn't have to suffer through the ceremony!
Best way to get married IMO, no stress, no pressure, no one to please but your SO. Not all families would go for it though.
When I did travel stuff, we did a few destination weddings, and while I didn't really know how many people they invited, the people that did actually go just called up and got it arranged, we didn't have anyone waffle or back out or anything like that, but it was only a couple so my experience might be slanted.
If I were invited I'd look at it simply as a vacation.. do I want to go there? Does it fit within my vacation planning in general (budget, timing, etc)? If so, then sure why not? If not, then I'd decline.
I'd still get them the same "level" of gift though, as I view my going as a vacation, not as an obligation.
I know some people that get very upset with people that have destination weddings, they see it as their friends/family making them pay just to participate in a happy time in their lives. And I can appreciate that view somewhat; if I was very close to the couple and couldn't afford to go I might be upset.. but I also want them to be happy and understand sometimes life isn't exactly fair. A general reception upon return could help mitigate that.