I think you know the problem, if you want to close a program, especially a game with Alt+F4, nothing happens.
Sure, I know every programmer can choose what will happen, if the user presses Alt+F4, but can you solve this problem with a litte other program or a registry tweak?
2 Answers
SuperF4 seems to do what you want, though it uses Ctrl+Alt+F4 instead. I haven't used it myself, so no guarantees on quality.
I'm not sure if it would help with the programs troubling you, but AutoHotKey is a free, well-known program for re-mapping keys, and it uses a script that includes commands for minimizing and closing windows, etc.
So, for example, if you wanted to be able to close windows using the shortcut, Ctrl+X, then you would include this command in your AutoHotKey script:
^x::WinClose, A
If you read the help pages or search for specific keywords in them, you should be able to find clues about how to add a different shortcut for that. I never had to read the full help pages; only ever searched for specific tips, found them, and taught myself how to use it through the months with a couple of minutes here and there, even though it felt confusing or overwhelming the very first time.
Hope this helps.