I did try that once and could not find the file! It seems it gets deleted and at some point the updater tries to find it again or something. I'm not sure. Either way, there were several files that could not be found, this was just the first in a chain so I decided to reinstall.
I have always had DCS as an exception, so unfortunately this did not work either.
I fresh reinstall ended up doing the trick.
SgtPappy's post in Access Denied - cannot delete file preventing update was marked as the answer