As a current Gamepass subscriber, the biggest shortcoming is wait times. There is nothing more frustrating than launching a game to be told that the servers are busy and the wait is 5-9 minutes. Then you just have to leave your TV or screen on that rocketship waiting screen because you are afraid you'll lose your spot in line.
PS
I am on Gamepass because I figured out how to get it free every month by getting microsoft reward points -
https://www.reddit.com/r/MicrosoftRewards/