


Steps taken:
Add non-steam game to steam: select the installer you downloaded from GOG as the file.
Change compatability setting to a defined version of proton. I used 8.0-5.
Run installer. This will install into your steam directory (for me this was /home/user/.steam/root/steamapps/compatdata/ - but run a search for "Freespace 2" if you are unsure). Note that the GOG version applies a patch directly to the proton files which is very important.
Launch game directly after installation when prompted. This should work out of the gate.
Now you just need to change the Steam shortcut of the installer you added to point at the game rather than the installer - use the directory setting you tried before, this time pointing to "Freespace 2.exe" and then run it (this ran a configuration program), and finally to FS2.exe (which now runs the game directly).
Done!

Using GOG version of the game: https://lutris.net/games/install/2675/view
The installed version is the Freespace 2 Open Mod script provided by Lutris. Looking up this game on WineDB will show that it's playable with WINE, so Proton shouldn't give any issues IMO.

Proton crashes with a error for me.

Game doesn't start with Proton GE 6.21-GE-2
"Freespace 2/Fred2 cannot be run from a drive root directory" is the error message. I'm too dumb to apply the registry fixes others provided.

Fails to start.
"Freespace 2/Fred2 cannot be run from a drive root directory" is the error message.
Fresh install fails to launch game with:
"Freespace2/Fred2 cannot be run from a drive root directory!"
You need to apply the FS2.reg file to the wine prefix for me the command was:
WINEARCH=win64 WINEPREFIX=$HOME/.steam/steam/steamapps/compatdata/273620/pfx/ $HOME/.steam/steam/steamapps/common/Proton\ 5.0/dist/bin/wine regedit ~/.steam/steam/steamapps/common/Freespace\ 2/Registry/FS2.reg
be sure to change the proton version as needed.

fails to load up out of the box.

Won't start.

Manually copy contents of steam/steamapps/common/Freespace 2/Registry/FS2.reg into steam/steamapps/common/Freespace 2/Registry/system.reg. Make sure to change the key path format from [HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Volition\FreeSpace2] to match the others already here.