You have to be very gentle with your controls, usually people tend to over correct.
I'm don't know what stick you are using but if you have a TM Warthog it might be worth removing the main spring inside the base so it's no longer self centering. It's much, much easier. Did that years ago and never looked back.