Virtual Reality

Here I keep my experiments with the Oculus Rift virtual reality headset. If you want to be notified when there is something new you can follow me on the Twitters: @Tamulur. If you have any questions, suggestions or feedback, please email me at tamulur@yahoo.com.

VR Review did some very nice videos of some of my experiments: Video 1 Video 2


Little Big Dungeon

LittleBigDungeon

This is primarily a test of switching between first- and third person perspective. Secondarily it is a test of dodging punches. Therefore it is best played standing; dodge with your upper body like a boxer. Explore the dungeon, switching between perspectives for better orientation and detailed view. But beware, a goblin roams the dungeon, the feared Iron Myke.

This is my first attempt to try to solve the problem of switching between 1st person and top down view in VR. The problem is that in 1st person view you look straight ahead, in top down you look about 45 degrees down. If you just switch from one to the other with a button press, you find yourself staring into the void or at the ground after the switch, which is jarring. Trying to make you look straight ahead for the top-down view by having the top-down view rotated 90 degrees in front of you like a map on a wall doesn’t work either, because you feel everything in the tiny world you see should be pulled down by gravity. The alternative solution tested in this demo (switching by head angle) works well enough for me so far.

Windows, Oculus Runtime 0.8 or newer: LittleBigDungeon_0.6.zip
Windows, Oculus Runtime older than 0.7: LittleBigDungeon_0.5.zip


Groundhog Station

GroundhogStation

Short experiment for the Razer Hydra. Not much replayability value, but for a while it is interesting to have NPCs come up to you and interact with you like real humans.

A train arrives at the Groundhog Station. Initially, everybody is just standing idly. You are put into the role of one of the people on the platform or the train and can move and act around. After a minute or so, the train leaves, the scene resets, the train arrives again, and you are put into the role of a different person. The previous person plays back what you did last time (similar concept to Time Rifters). This way you can bring the scene to life by filling each person’s role and making them act out different things, like greeting each other with a handshake, or one person waving another goodbye.

Windows, Oculus Runtime older than 0.7: GroundHogStation_1.1.zip


Let’s Dance


Short experiment with delayed mimicry for the Razer Hydra. You need to be standing for this experiment. The NPCs mimic your dance moves, each with their own delay. The different delay is supposed to make them seem less automatic. I made this experiment to find out whether it feels fun to dance when NPCs dance with you in the same way.

Update: I was just dancing with the woman in the red vest and at the very moment when our virtual hands happened to touch, my real hand accidentally touched the Hydra base. My first flash of presence. Also, I almost crapped my pants.

IK system used: FinalIK

Controls:

Left Hydra:
Button 4: Recenter view
START: Recalibrate hands

Windows, Oculus Runtime 0.8 or newer: LetsDance_1.9.zip
Windows, Oculus Runtime older than 0.7: LetsDance_1.8.zip


Coffee without Words

CoffeeWithoutWords Quick test to experiment with procedurally generated NPC eye movement.

Some of the papers used:

I summarize what I used from these and similar papers in this post. If you want to save time and don’t want to code that yourself, I made a Unity asset for this.

Windows, Oculus Runtime 0.8 or newer: CoffeeWithoutWords_1.6.zip

Windows, Oculus Runtime older than 0.7: CoffeeWithoutWords_1.3.zip


Movement Experiments

MovementExperiments

Miniexperiment to test different movement systems to reduce simulator sickness.

Keyboard controls:

Space: Recenter view

1: Standard movement mode
2: Canvas movement mode
3: Third person movement mode
4: Stepwise teleportation mode
5: Stroboscopic movement mode

Canvas mode: When you move or turn, a canvas that is anchored to your avatar’s body is partly faded in. The idea is that this makes the VR world look like it is projected onto a screen and the screen provides a frame of reference: when you turn, the world doesn’t turn around you, but only its projection on the screen turns. The screen is still relative to your avatar body, so your visual and vestibular system don’t get conflicting input. Press G to cycle through different canvas textures.

Third person mode: To move, keep the right mouse button pressed. The environment will be shown in an ambient occlusion rendering style. With the right mouse button pressed, use WASD to move your avatar and the mouse to turn it. Your viewpoint itself doesn’t move, but you can keep looking around with your head. To teleport your view into the avatar’s new position, release the right mouse button.

Stepwise teleportation mode: Keep the right mouse button pressed to enter movement mode. Look where you want to go; you will see a ghost avatar at the target point and a path leading there. You can turn his look direction with the mouse. Release the mouse to move; you will be teleported there in steps.

Keys:
7,8: de-/increase step size
9, 0: de-/increase step duration

Stroboscopic: Stroboscopic view when you move or turn. It strobes by showing x frames in a row, then showing darkness for y frames in a row. To decrease/increase x (shown frames), press 7 and 8. To decrease/increase y (black frames), press 9 and 0.

I get less simulator sickness in both the Canvas and the 3rd person modes. The canvas mode is a bit more immersive, but the 3rd person mode lets me assume the avatar’s identity more.

Windows, Oculus runtime 0.8 or newer: MovementExperiments_0.8.zip
Windows, Oculus runtime older than 0.7: MovementExperiments_0.7.zip
Source: GitHub


Subway Car

SubwayCar First game mode: When you walk around the subway car, people briefly look at you when they notice you. If you walk up to and keep staring at someone, he or she will eventually stare back, the rest of the world fades into a different rendering style and time slows down. This is supposed to intensify the effect of a computer character staring at you. There are other game modes as well, they are more or less about how you experience reality. Cycle through game modes with G. To show info about the current game mode, press H. But first try to find out what it does on your own.

Windows, Oculus Runtime 0.8 or newer: SubwayCar_1.3.zip
Windows, Oculus Runtime older than 0.7: SubwayCar_1.2.zip


Three Voices

ThreeVoices Mini-experiment for binaural sound, using the “cocktail party effect”, the phenomenon that we can concentrate on one voice out of several. This experiment tests whether the effect is stronger if the voices are generated binaurally (via the 3Dception plugin) than if they just use Unity’s standard 3D sound. The demo’s first phase plays three voices with Unity’s standard 3D sound. The second phase plays them binaurally. Start each phase by stepping into the light. You need headphones for the binaural effect, preferably good ones. You should turn off any surround-sound simulation you might have activated on your computer (CMSS 3D, Razer Surround etc.) I find it much easier to concentrate on one story in the binaural phase.

Windows, Oculus runtime 0.8 or newer: ThreeVoices_1.5.zip
Windows, Oculus runtime older than 0.7: ThreeVoices_1.4.zip
Mac, DK1: ThreeVoices_Mac_1.2.zip


 KnockKnock

KnockKnock This demo experiments with binaural sound effects in VR: Some sound effects, like the footsteps, are generated with the 3Dception Unity plugin. Others, like the knock or the whisper, are recordings from a real binaural setup and are played back in this demo when the direction of the player’s head coincides with the recording. You need headphones for the binaural effect, preferably good ones. Credits:

Binaural knock: audiocheck.net
Binaural whispering woman: HeatherFeather
Binaural sound engine: 3Dception

Windows, Oculus Runtime 0.8 or newer: KnockKnock_0.6.zip
Windows, Oculus Runtime older than 0.7: KnockKnock_0.5.zip
Mac, DK1: KnockKnock_Mac_0.4.zip


Ambient Occlusion Room

Screenshot_AOonlyScreenshot_FullTextureScreenshot_Mixture

Miniexperiment. Idea: Rendering a scene with ambient occlusion only (no textures, a kind of black&white, lighting-independent rendering) makes the scene’s spatial structure super-parseable for the brain’s visual perception system. You immediately get the complete Where information of the whole scene just from your peripheral vision. But to get the What information of individual objects you need to see their texture. But if we let all objects show their texture we don’t have the super-pareseability of the ambient occlusion anymore. This demo experiments with having most of the scene in ambient-occlusion, and only the objects in the center of view or close to you rendered with texture.

Windows, Oculus runtime 0.8 or newer: AmbientOcclusionRoom_0.4.zip
Windows, Oculus runtime older than 0.7: AmbientOcclusionRoom_0.3.zip


Time of Statues

Screenshot This is a short experiment to test relaxation/time perception in VR. It is based on being moved very slowly through the environment. Go to the statue, look up at her until she notices you, then walk onto the platform and wait for her to lower her hand. Go onto the hand. From then on the demo is non-interactive. Just sit back and enjoy the music. For copyright reasons most music is streamed from YouTube. If you don’t have a good internet connection the music might not play at the right time.

Windows, Oculus Runtime older than 0.7: TimeOfStatues_1.2


Ask Socrates

AskSocrates_Screenshot This is a quick little experiment to try out how VR can help with brainstorming. When thinking about an issue, it often helps to look at it from the viewpoint of someone else. This demo lets you alternate between being yourself and being Socrates, who keeps questioning everything you say, until some great insight emerges.

Start the demo and type whatever issue is on your mind. Keep it short, maybe one or a just a few sentences. Whatever you type appears on your notepad. Then type Enter. You viewpoint will change to Socrates, and what you wrote appears on the blackboard. As Socrates, type whatever cynical/critical remark the grumpy old philosopher would have come up with when confronted with such youthful naivety.

Continue the dialogue until either you are too confused to go on or you have a revelation. Then press Escape to quit the demo. A text file containing the whole dialogue will be saved in the demo directory.

May you be enlightened.

Windows, Oculus Runtime 0.8 or newer: AskSocrates_0.5
Windows, Oculus Runtime older than 0.7: AskSocrates_0.4
SourceGitHub


Pumpkin Punch

I created this small game with James and Alex to try out punching with Razer Hydra controls and the Oculus Rift headset.

Windows, DK1PumpkinPunch_02.zip


Avatar Experiments

A quick test of seeing your avatar in a mirror with Oculus Rift and Razer Hydra, based on Dessimat0r’s demo GirlMirrorLook. There are several avatars in the scene, you can switch between them. If you have Hydras you can control your avatar’s hands, but the demo runs without them as well.

Windows, DK1AvatarExperiments_02.zip
Source: GitHub

32 thoughts on “Virtual Reality

  1. Heather Feather

    Hello. I tweeted you, so I hope this message or the tweets find you. I was notified about this blog via someone on twitter. I saw you sampled some of my audio, and I was very very honored. I wanted to thank you. VR is something I am really interested in, and seeing it (if only briefly—I can’t download the full demo as I am not on my own comp) got me really amped for how immersive things can be in the not so distant future. Anyway, I won’t talk your ear off, but I wanted to express my gratitude. Thank you. That was a really cool surprise, and it made my night.

    Sincerely,
    Heather Feather

    Reply
    1. Tore Post author

      Thanks Heather! I really love your videos, and I think there is a lot of potential for integrating ASMR into VR.

      Reply
  2. Ross

    I was very interested to download the Subway software, having recently got hold of a DK2. As a sufferer of social phobia, I have been looking for software that could potentially provide a form of exposure therapy, but it is not possible to buy commercially, and those that are available usually include a therapist fee.

    The subway program comes close to being able to activate some of the fears needed for exposure, however I wondered if you have considered or even made a version where your computer characters actually respond, perhaps in negative ways? As this is so often the crux of social phobia – the fear of experiencing negative social evaluation, you could potentially have a very powerful piece of software here.

    For example, the guy with the computer becoming very “headmasterly”, or one of the attractive women scolding you for looking at her (assuming user is male), or the children being rude etc. I appreciate that would probably mean a ton of sound assets and dialogue trees though, which would be time and money consuming to generate.

    Reply
    1. Tore Post author

      Hi Ross,

      I would love to experiment with a more full featured social interaction simulator. Unfortunately for now I do not have access to facial animations or even custom body animations, let alone custom voice recordings of different characters. As soon as tools for that become more easily available, I will definitely play around to find out how to make computer characters more life-like and how they can make us feel like there is “someone there”.

      Reply
      1. Ross

        Hi
        Its been a while since I wrote, but I wanted to say how impressed I am with “coffee without words”. Like the subway car, such impact! I am still hunting around for a “conflict and rejection simulator” 😀 The work you are doing is fantastic. It felt like the final coffee on the brink of a horrible break up! If only she could speak.

        Also you made me want a latte 🙂

        Reply
    2. Georgina Kelly

      Hi Ross, I too am interested in VR from a therapeutic perspecting, so would be very interested in any feedback you have about your experiences with VR. Was existing technology sufficient to give you a truly immersive experience? Did you find simple exposure was sufficient to redeuce you fear over time? what was lacking? I am new to VR and very eager to explore, what suggestions do you have? what programmes and what attributes of those programmes improved your “recovery”? what was most useful to you? Thanks, Georgina Kelly

      Reply
  3. dave

    Is it possible to get the subway car simulator working properly on the DK1? (It does work but the pictures displayed look wrong and it hurts the eyes looking at it.)

    Reply
    1. Tore Post author

      Hi Dave,

      Subway Car works fine for me on the DK1. Have you downloaded and installed the latest runtime 0.4.2 from Oculus? Does your monitor run in extended mode?

      Reply
  4. Pingback: Some of the best Oculus Rift DK2 demos to show your friends | KitGuru

  5. James

    ThreeVoices crashes on startup:

    “”the direct3d 11 mode is not supported in this configuration. please use a different display mode, a different graphics API, or rebuild the application with a new Unity version.”

    This is in extended mode. In direct mode it just flat won’t start.

    Reply
    1. Tore Post author

      Does your graphics card support DirectX11? If not, you might try to force DirectX9 mode by creating a shortcut to the exe and adding -force-d3d9 as command line parameter.

      Reply
  6. Brent

    Please keep the experiments coming. You are exploring areas of VR I did not realize were so important. Very imaginative thinking here. Thank you for motivating me to make something great.

    Reply
  7. Amy Dentata

    These are some of my favorite experiments I’ve come across so far using the Rift. There is a lot of potential to make games that provide experiences that aren’t otherwise possible, especially with regards to social interaction. I see eye contact—and body language involving it—as an unrecognized powerful tool. If later versions of the Rift include eye tracking/face scanning then it will become even more powerful. Thanks for all the really inspiring/insightful experiences here.

    ~ Amy

    Reply
  8. Drew

    Hey Tore,

    I really love your demos, especially Groundhog Station. It’s the closest I’ve come yet to feeling present in a virtual space. Thanks so much for making these excellent demos, and for publishing them here on your site.

    I do have one bit of feedback, however 🙂 In Groundhog Station, it can be frustrating to spend only seconds as each character before the screen fades to black. Would it be possible for users to pause the motion recording and transitions between characters, and just explore as the station as their current avatar? Kind of like “Avatar Expirements,” but with the DK2’s positional tracking. This would add a great deal of replayability!

    Reply
    1. Tore Post author

      Hi Drew,

      Thanks for your feedback! Switching between avatars on the fly and explore the environment is a good idea. I think a more interesting scenario than Groundhog Station would be better though, one with mirrors and cameras maybe. One of my future planned experiments might be appropriate for that, if so I will implement it.

      Reply
  9. Pingback: The Power of a VR Experience - dSky/Rift: the VRFAQ

  10. Moritz

    Your demos are awesome, and very interesting!

    But i have a mayor complaint! Why is my sad/angry girlfriend from “Coffee Without Words” in the subway car, with that dude’s arm around her! Did she get break up with me that quickly, and ran off with a new guy? 🙂

    But seriously: I thought it felt weird/shocking seeing her there, in a different context and situation, which is a testament to that strange feeling that Coffee Without Words causes in many people, which in no small part must be due to your eye animation. Great job!

    (but please tell us at least WHY she’s mad at all of us? 😉 )

    Reply
  11. Artist3d

    Fascinating convo with Socrates…

    You:
    Consciousness and awareness is there a difference?

    Socrates:
    Yes indeed, one is the container of the other.

    You:
    I See and does one or the other survive death?

    Socrates:
    It is likely that the dependencies of consciousness on input from the physicality of sense impressions of the body, that is something we are aware of, would be…

    You:
    yes I understand, unlikely to survive the demise of the body. And awareness?

    Socrates:
    Well strictly speaking awareness is the sum total of the experiences of one’s consciousness yet mysteriously of a never born and never dying nature… or so they say (Tibetan Mystics).

    You:
    So it goes on after death…?

    Socrates:
    It is a reasonable assumption to make if it is not tied to the brain or sense impressions but having always been the background witness to it all… that is aware of one’s life experience.

    You:
    Thank you Dr. Socrates this has been a dreamlike experience with myself so I would obviously concur at this point.

    Socrates:
    Perhaps we will chat again Paul, good to see you. Keep dreaming, I ‘ll be here the next time.

    You:
    Bye now

    Socrates:
    goodbye.

    Reply
  12. Jordan

    Awesome look at things here. It’s amazing what’s being integrated here. That Socrates game sounds interesting especially. Would love to give these a shot!

    Reply
  13. Nicki

    I am doing some research on the history of VR and am looking for great archival content. Does anyone have suggestions?”

    Reply
  14. James Morgan

    My girlfriend loves the dance demo, but she got into some motion-sickness afterwards. Unforunately I guess it will take some years of development to find a proper solution :/ Funny thing is that some Sony guy just said that Project Morpheus wont have games with 50 hours of gameplay, due to VR causing nausea: http://data-reality.com/sony-project-morpheus-to-work-only-for-short-sessions/

    However, thank you for doing this Tore, I love your demos as well! Keep it up 🙂

    Reply
  15. Oviah

    Just the pictures make you want to go out and buy a headset. VR will take gaming and immersion to a level that might just trick the brain into thinking it’s real 😉

    Reply
  16. bob

    Your work is really interestsing, unfortuanetely coffe without words (1,5 with sdk 0,7) doesn’t start on my rift but on the desktop. Any way to fix it?

    Reply
    1. Tore Post author

      Coffee without words requires the 0.8 Oculus runtime, I adjusted the requirements documentation.

      Reply

Leave a Reply

Your email address will not be published.