Ah so Im not going mad, this has been happening to me
I completely removed all my joystick and throttle settings and then reassigned them all, seems a lot better but theres still some pull at times.
This was in 2.0 at first but seems to be the same in 1.5 now.
Its not my joystick, as I say I removed all controls, axis the works, tried with joystick disconnected just using keyboard and its the same. Even with control display (right ctrl and enter) showing zero movent from any controls its still been happening.