Can I elevate privileges if an application installs to a user writable folder, but then deletes itself?

If an application installs itself into a user-writable folder, but then deletes itself after installation, is there a way to exploit that to elevate privileges?

So say this application installs to C:Temp and BUILTINUsers group has full access (F) to the directory, I assume there is a way to elevate? I tried creating a payload with the same name and putting it in that folder, but the installer just ignores it and creates a new executable.

The problem is that the installation happens pretty quickly, so I’d have to overwrite the binary during install. Is there a way to overwrite it with PowerShell during the installation process?

Edit: the application is installing as SYSTEM

Source: Windows Questions