OSX Notifications moving Desktops around.

So there is this really annoying bug/feature in OSX that is killing me.

Imagine you are working on Desktop 1 (Browser) and Desktop 2 (Xcode), and Messages is open on Desktop 3. You are currently on Desktop 1 and you get a notification from Messages. When you click the notification OSX will bring you to the Messages app as expected. But, it will do this by moving Desktop 3 to where Desktop 2 was, basically swapping the two desktops. So when you go back to working on Desktop 1 and you slide over to where you expect to find Desktop 2 you will instead find Desktop 3.

Do any of you guys know of a setting that will disable this so that OSX will bring me over to Desktop 3 from Desktop 1 instead of moving Desktop 3 to where Desktop 2 was? I've tried googling it and searching through my settings with no luck..