There are two ways to aviod this.It's normal and caused by ground stablize when you are in point track or area track which initially track the ground closer to you.
1.when right MFCD (TGP page) as SOI and, and TMS aft short and you engage INR track(When in Inertial (INR) mode, the TGP will remain fixed on a geographic reference point.).
By TMS aft short,you will see a INR on your TGP screen where you normally see a 'point' or 'area' .when in INR track,anything obuscure your sight won't affect your TGP.
2.Make a markpoint at where you want TGP to stay,make the markpoint as steerpoint,and slave you TGP on it.