I followed this guide to reset permissions and it seems like that did the trick.
You can reset custom NTFS permissions applied to a file or a folder in Windows 10. After performing this operation, custom access rules will be removed.
winaero.com
Thankfully there's the command that applies it to the folder, sub folder and files so I didn't have to do it too many times manually.
Edit: Lol, after reboot it applies the permissions to how they were before the fix... What the hell
These are the permissions after reboot where the issue persists:
These are the permissions after resetting them with that command where the issue is resolved:
Why do they change back? Log out/log in doesn't cause that, shut down/restart does. Googling comes up with an OneDrive bug, I don't use it at all though.
Edit: Ok finally fixed. I had changed the default install folder, I thought that only applied to Windows Store apps (after all, programs with setup files always ask and still have C:\ as default iirc) as the default install location. I changed it back to C:\ from the windows settings > system > storage thingie, re-did the whole process and after another reboot it seems to retain the proper permissions. Just weird that it only did this the last month or whenever, I've changed that setting there since at least last year (it shows modified date) to reinstall Phantom Dust free from the Windows Store. Dang, stupid shit.