In short, as the game runs, probably loading mesh and similar data, occupies a lot of VRAM, never completely freeing it. I arrive in having the full VRAM occupied and that decreased the fps, probably halving it, leading to input lag and such. From the sensors, I see 6056/6144MB of VRAM used continuously.