Grigor Todorov

Somnai

This was a freelance project for dotdot.london.
I worked on the Unity application for the Fantasy VR experience, a location-based VR installation where groups of six participants wear VR backpacks and explore a series of fantasy scenes together.

Fantasy VR is a networked, multi-user experience. Participants can interact with each other and with the virtual world around them. It is also a multi-sensory installation, using scent, wind, heat, physical props, and changing floor surfaces to create a deeper sense of immersion.

Hardware:

    We used more than 38 OptiTrack cameras, along with OptiTrack HMD trackers provided by Target3D. For hand interaction, each headset had a Leap Motion sensor attached to the front. The avatars were driven by FinalIK, working together with Leap Motion and OptiTrack tracking data.

Somnai:

    The six participants were guided through the experience by an actor. In the virtual world, the actor was represented by a glowing sphere called "Somnai", which was controlled using an OptiTrack active tracker.

Networking:

    For networking, we used UNET. Working with six VR backpacks could be challenging, so I set up a remote connection system. Each backpack ran a client application that automatically discovered and connected to the server. The server ran on the same machine connected to the motion capture switch and OptiTrack software.

Scents, Wind, and Heat:

    These elements were created by the amazing Grace Boyle from The Feelies, adding another layer of immersion to the experience.

Sensory Elements:

    A wooden bridge, multiple props, and different floor surfaces were used throughout the experience to enhance the participants' sense of presence.

I'd also like to share a few behind-the-scenes pictures.

First test. The volunteers formed a chain by holding each other.
Going through a virtual cave tunnel
First test with 6 people. Only 5 backpacks worked that day. The Oculus software on the last backpack was not working properly.
Rehearsals and training. Here you can see Somnai guiding the users through the bridge.
Chilling in the Fantasy Forest scene
It all started with an empty warehouse... The whole Somnai experience takes place in a 20,000 square feet warehouse. 
OptiTrack cameras. The fantasy experience uses 38+ OptiTrack cameras!
OptiTrack active puck. An active tracking puck was used for the Somnai sphere.
Oculus, OptiTrack and Leap Motion. Oculus HMD was used with an OptiTrack active tracking clip and a Leap Motion tracker for user interactions.
6 backpacks. My temporary labelling system was a disaster and after the first test most of these labels fell off.
Preparing the backpacks for the first test. It was a daunting task to keep 6 backpacks working. I set up remote access to all backpacks. I also implemented a battery level UI in the Server app to make it easier to detect when a backpack battery needs changing.
Scent machine. Scent, heat and wind machines were used through the experience. 
Batteries. The backpacks require regular battery swaps. The experience is very heavy on the machines and the battery drains really fast. I impemented optimisations to increase the battery life while the application is not active. 
Server application. Backpack clients connect with the server automatically via network discovery. The server is used to configure the backpacks and the world. It also has a few features to make the life of IT technicians easier, such as callibration tools, battery life indicator, saving values between sessions etc.
Huge Mushroom prop. A giant mushroom prop that's soft on touch. This prop is alligned with a virtual mushroom with the same shape.
Wooden Bridge. The first thing the users have to do is go through a wooden bridge. The bridge has an exact representation in VR.
Floor. Different floors were used for different scenes to provide another layer of immersion.
Another prop. This prop has squashy balls the user can interact with.
More props
More props. 6 mushroom props in total were used in the experience.
Backpack harness. These made it easier and faster for users to get in and out of the backpacks.
❮
❯