Author: Ted Thomas

Implementing VR Eye-tracking and foveated rendering in Unity with the VIVE Pro Eye

Eye-tracking is set to become an important VR hardware feature over the next few years, offering many benefits including greater immersion, new analytical possibilities, and better performance.  With the recent release of the Vive Pro Eye, sophisticated eye tracking is now available built-in to a commercial VR headset for the first time. In this blog post I will show you the steps required to set up eye tracking with the Vive Pro Eye using Unity.  

First, create a new Unity project using the 3D template. For this example we are using Unity 2019.2.0f1. Once the project has finished setting up, import the SteamVR Plugin from the Unity Asset Store (https://assetstore.unity.com/packages/tools/integration/steamvr-plugin-32647). Once  imported, accept the recommended project settings when prompted. 

Next go to the download section of the HTC developer website https://hub.vive.com/en-US/download (you may need to sign up to gain access). From here download and install VIVE_SRanipalInstaller_1.0.3.0.msi.  After this, download SRanipal_SDK_1.0.1.0_Eye.zip and extract the contents. Within this folder you will find a Unity package file. Double click it and Unity will prompt you to import into your open project. 

After importing the package open the EyeSample scene contained within the ViveSR. This scene will allow you see you the eye tracking works in a simple environment. If the tracking doesn’t seem accurate you can launch the calibration settings from the Unity game window when play mode is active. For additional information on how to use Vive eye tracking for your project, checkout the PDF in the SRanipal_SDK_1.0.1.0_Eye folder.  

When using a headset that features eye tracking, we can implement foveated rendering to increase rendering performance. Vive provide their own implementation of foveated which is available on the Unity Asset Store (https://assetstore.unity.com/packages/tools/particles-effects/vive-foveated-rendering-145635). After importing the package, go to the Unity Player settings and add “USE_SRANIPAL” in the Scripting Define Symbols field. Then simply add the ViveFoveatedRendering.cs script on to the VR camera in your scene. To ensure that foveated rendering is working correctly, add ViveFoveatedVisualizer.cs to your camera. You should see the foveated targets following your eyes as you move them. 

Using VRTK 3.3 with SteamVR

VRTK with Unity 2017.4

This guide describes the process of setting up VRTK to work with Unity using Steam VR.

Compatible headsets for this guide-

  • HTC Vive, Vive Pro
  • Oculus Rift
  • Windows Mixed Reality Headsets

This guide is also available as a YouTube video:

VRTK is a VR Toolkit for rapidly building VR solutions in Unity. As of November 2018, the most recent of VRTK found on the Unity Asset Store is no longer compatible with the latest versions Unity, Steam VR for Unity (Steam VR 2.0) or Windows Mixed Reality headsets.

To get around these limitations we will be using the latest version of VRTK, and older versions of Steam VR and Unity.

Step 1– Install Unity

VRTK works most reliably with the Unity 2017.4 LTS releases. The easiest way to install Unity is by downloading the Unity Hub, available here https://store.unity.com/download?ref=personal

From within the Install section of Unity Hub, download the most recent version of 2017.4 LTS, which at time of writing is 2017.4.14. If there is no 2017.4 version available, download directly from the Unity website here: https://unity3d.com/unity/qa/lts-releases After Unity has installed, create a new project.

Step 2 – Install VRTK

Download VRTK as a zip file by using the green download button on the VRTK GitHub page, https://github.com/thestonefox/VRTK . From within Unity, right click the ‘Assets’ folder and select, ‘Show in explorer’. Replace the ‘Assets’ and ‘Project Settings’ folders with the versions from the VRTK zip folder. Clicking back on Unity will now import VRTK into your project.

Step 3 – Install Steam VR

Download the Steam VR 1.2.3 Unity package by visiting this link https://github.com/ValveSoftware/steamvr_unity_plugin/releases/download/1.2.3/SteamVR.Plugin.unitypackage In Unity, import this package by clicking on the Assets menu > Import Package > Custom package and selecting the Steam VR Unity package.

Step 4 – Configure VR SDKs

Configure the necessary VR SDKs within Unity by clicking on the Edit Menu > Project Settings > Player. From here click on XR Settings and make sure ‘Virtual Reality Supported’ is ticked. Now add Open VR to the list of VR SDKs if it is not already there. If you are going to be using an Oculus headset, add the Oculus SDK here too.

Step 5 – Testing

Unity should now be setup correctly for use with VRTK. To test, make sure your VR headset is connected and open one of the example scenes found in the VRTK/Examples folder. Click the Unity play button and you should be able to interact with the scene.