I was very impressed that this is possible. I was trying to do a test with NVIDIA GRID in vDGA (passt-hrough) mode on a Windows Server 2012 R2 XenApp 7.5 and a vSphere 5.5 host with a GRID K2 core assigned.
This is a direct RDP connection from Windows 8.1 client to the server.
The performance is amazing for RDP but look at the bandwidth consumption and CPU. It doesn’t compare with HDX 3D Pro but still, it’s great to see it working!