How to capture through the lens photos on VR/XR headsets

Capturing and portraying what a user actually sees through the lens when wearing a virtual or mixed reality headset (HMD) can be very difficult. Screenshots or captures via software implementations (such as the ‘Record’ function in Varjo Base) do a great job at demonstrating the intended content, but do not account for the effect that the specific HMDs have on the experience, missing for example the lens quality or field of view.

Nothing compares to the immersive experience of wearing the HMD in-person but taking a photograph inside the headset is perhaps the next best thing. Framing the image from the perspective of a user captures the combined experience of the visual content and the hardware’s delivery of said content.

However, capturing a true-to-life image ‘through the lens’ does take a certain amount of expertise. When done badly, imperfections such as chromatic aberration or soft focus generated from the use of the camera, can give an unwarranted poor representation on the actual visual fidelity of the device in question.

The ultra-high visual fidelity of Varjo HMDs is unparalleled in today’s VR/XR market, and care should be taken to do justice to the visceral experience of VR/XR with Varjo. Below are my tips on taking a clean photograph through the lens of Varjo HMDs.

Photographing Varjo Native applications through the lens of VR Headsets

• In general, the better the camera quality, the better the results for through the lens VR photos.
  • ‘ISO Invariant’ cameras are optimal, as they do not add noise to image with a high ISO.
  • I personally use the ‘Sony RX100V’ with the ‘Sony Image Edge Camera Remote’
  • Panasonic GH5 with 7.5mm lens is also a good option
• It is best to take pictures via the HMD’s left eye lens. Most cameras are designed for right hand users, so the camera handle is on the right side of the camera, which makes for easier positioning.
• Using a good tripod can help to attain a crisp image.
• Using an external camera trigger is useful to prevent accidental camera shake when taking the picture (Some cameras can be controlled with a mobile phone)
• Ideally, use a PC camera remote application. This will help a lot with manual focusing since you will get a much larger image preview on a PC monitor than on a camera display.

1. Open “Analytics window” from Varjo base
2. Choose the target what you want to take a picture of through the lens of VR headset. Since you will be taking the picture from left lens it is easier to align target with right eye closed.
3. When the target is in the center of the left eyes view, activate “Lock headset pose to current location” in the analytics window. Now the image is locked and moving the headset does not have affect what you see in the headset.
4. In the ‘Analytics view’, activate the ‘Show foveated rendering area’ option and ‘Move gaze with mouse’. With the Varjo Aero, activating both ‘Show foveated rendering area’ and ‘Move gaze with mouse” is fully required to get full resolution on your target area.
5. Make sure that ‘Automatic eye tracking calibration’ is set to ‘Off’. If it is on, there is a possibility that when you put your camera in front of the left eye lens, the gaze calibration will be wrongly triggered.
6. When ‘move gaze with mouse’ is activated, you will see a yellow rectangle following your mouse cursor in analytics window.
7. Move the yellow rectangle with the mouse so that it covers your target. Ideally your target should be in the center of the view.
   a) If you are using remote camera control software on a PC, you should use use ALT + TAB to move away from analytics view to any other window that opens on top of analytics window (and covers is completely.) This is to ensure that you don’t accidentally move the yellow rectangle in analytics window and move the foveated area away from your selected target.
   b) You might notice that if gaze calibration is seen in analytics window, the gaze dot is not in the center of the view. This is because our eyes naturally converge towards the center and that is how we get stereoscopic vision. A VR headset does the same thing by default. But if you would try to put the gaze center point dot in the center of your camera image you will get heavy chromatic aberrations because your camera is not pointing directly to the optical axis of the lens. You will get better results if you place your target in the center of the view.
8. Adjust your camera settings until the exposure looks correct. You need to take test shots and make iterative adjustments to find the correct settings. Here’s what works in my setup;
   a) Use full manual mode if possible. Usually, it is called ‘M’ mode
   b) Use a small aperture like F8-F11
   c) Use a long exposure time 1/60 – 1/15
   d) Use ISO 800 – ISO 3200 or “ISO AUTO” mode
   e) Enable image stabilization if your camera has it
   f) Enable manual focusing in camera
   g) Use RAW + JPG mode to get best resolution and editability
9. Place the headset on a table and place the camera in front of the left lens and focus manually until the image is as sharp as possible.
   a) You should point the camera directly to the center of the left lens
   b) If you see chromatic aberrations in the center, it means that your camera is not properly aligned. You may see some aberrations or blurriness in c) the edges of the view since it is very difficult to have a perfect alignment.
   d) If you see dark horizontal stripes, it means that your exposure time is too short
   e) If you see a moiré pattern you can try to get rid of it by using smaller aperture (meaning larger aperture number.)
10. If you don’t want to place the headset on a table and you want to take the pictures more ‘free hand’, you need to have ‘faster’ settings for your camera
    a) Make sure that you hold both the headset and camera as stable as possible
    b) Prioritize for a shorter exposure time like 1/60 – 1/125
    c) Use a wider aperture such as F5.6 – F8
    d) Adjust the ISO so you get correct brightness levels
    e) Enable autofocus
    f) Use the self-timer feature in camera

Tips on how to capture through the lens VR photos from non-native apps such as SteamVR apps

• With SteamVR apps, you cannot stop the headset tracking so you must shoot ‘free hand’
• SteamVR apps don’t have foveated rendering so adjusting the relevant settings in the Analytics window will have no effect.
• For the best results, you should set the “image quality” to the highest option in the Varjo base software. This makes the application rendering quality better and as a result, the refresh rate may get slower. For through the lens VR photography, this may help to get better photos since when the framerate is lower, the image is not so ‘alive’ in the headset, and the reduced motion increases the chance of a clean take.
  • Use auto focus
  • Use full manual mode (M) if possible
  • Enable camera image stabilization if available
  • If adjustable, select the widest position on your lens
  • Don’t zoom in. Your hands will shake a bit and zooming in exacerbates the shakes.
  • Use aperture F4 – F8, adjusting for how much the required higher ISO values add noise to the images and how much moiré becomes visible
  • Use short exposure values e.g. 1/60 – 1/125
  • Adjust the ISO for correct brightness or use AUTO ISO
  • Use the self-timer function on the camera

Follow Varjo on LinkedIn