WSL Ubuntu Error: 0x80070050 The file exists. On Installation

  ubuntu, windows, windows-subsystem-for-linux, wsl-2

I recently renamed my user folder on windows and changed the subsequent registry keys to reflect the new name.
Everything works great besides Ubuntu, I tried following a couple of forum posts to resolve the issue, but nothing worked so I decided to reinstall Ubuntu. This is where I was met with the error:
Error: 0x80070050 The file exists.
I tried reinstalling WSL but it did not work. Any ideas? WSL error message screenshot

Source: Windows Questions