It does remain where it is when the Up or Down-arrow is released via keyboard.
The issue is with the pitch-input re-entering when pitch input is full aft. Irrespective of keyboard or Flight-Stick, auto-centering is by design/function of the AoA and Ny limiter. In other words, even if you were flying with a broom-handle, the IFCS WILL move your pitch from full aft to nearer centre.
Disengage the AoA and Ny limiters and full aft pitch remains in both keyboard and flight-stick.