Hovering with a harrier is a massive matter of practicing. Same with Helis as the Huey.
My friend flies in VR and he says that it is much easier as you can sense much earlier that the airframe tends in a certain direction, of course giving you faster reaction to counter that movement. I personally fly with trackIR. Without it I would probably never get a hover done.
Joystick sensitivity is a big thing. Giving too much curvature/deadzone results in far too much movement of Stick and Rudder for relative small corrections, while too small values result in almost instant Oszillation and loss of control.
I have a Thrustmaster Warthog combined with a T16000 Rudder, my curvature of the pitch and roll axis is at 20 and a deadzone of 3. The curvature of my rudder is at 35 and 5!!!. Sounds much but this relatively shitty pedals combined with my absolute lack of sensitivity in my feet/legs make everything below that hard to controll the turns.
After about 30 to 40 landings on the Tarawa I am able to bring her down quite on point on a good day.
I built a mission with some airshow cones as a slalom course on a runway to practice traversing from hover to controlled forward/backwards flight and strafing with helis and the AV-8. You can of course also use landmarks as a reference. Works pretty good.