When a game loses focus, it gets removed from list in pavu, so I had to use a script to change output sink from my headphones to Index.

https://gist.github.com/Programie/b50106cd2e66b3e474884f057507759a

Run pacmd list-sinks to get the name, then run

sleep 5 && ./pa-app-sink-switcher.py alsa_output.pci-0000_10_00.1.hdmi-stereo-extra3

where the second parameter is your sink name. Quickly switch back to the game, after few seconds audio output should get switched.