This was perhaps one of the most influential classes till date. In this exciting one week course we learned about the fundamentals for crafting an immersive experience through a lot of experimentation. This experience could live anywhere starting from the physical world to the pure digital world, and especially in between them through mixed realities. We learned about the different kinds of immersions and also the various techniques that can be used to prototype them. At the end of the week, each team also built a functional prototype of an immersive experience based on the learnings from the class. Moreover, we also had a lot of fun with the AR and VR toys our teachers brought with them: the holo lens and HTC Vive and it helped us understand a lot better about the virtual universe.

The teachers who flew in for this course were originally from the Microsoft Holo Lens team and carried with them loads of experience in working with mixed reality technologies and crafting immersive experiences. James and Josh had a great presence in class and also shared amazing examples from their work as well as others that served as good inspiration. James Tichenor is an architect by training and had received a Masters in Design and Computation from MIT School of Architecture. He works in the intersection of design, electronics and interaction.

Joshua Walton is an interaction designer who has a background in visual design and works in between software engineering and interaction. He is a great visionary and a science fiction aficionado, he had also me a lot in crafting my final proect: Lumen. Josh and James met and worked together at the LAB at Rockwell Group in New York where they experimented with various mixed reality experiences soon after which they were invited to Microsoft to work on the Holo Lens project.

In this week, we started off with a bunch of lectures from James and Josh that explained about what is it that makes an experience immersive and the different kinds of techniques that can be used to simulate such an experience. After two and a half days of exciting lectures and some brainstorming about our project, we had time from Wednesday evening till Thursday night to built our prototype for an immersive experience.
Apart from this, the teachers also set up a virtual reality station using HTC Vive at the corner of our class and also handed out a few holo lens kits, so that we can test out these technologies and also use it for prototyping experiences.

This was one of those classes where my brain turned into a sponge to be able to absorb all of the amazing lessons we were learning, and for the sake of summarising them, I will talk about a few of the ones that really stood out:

1. Embodied cognition
   Through the lectures from James and Josh, we got to learn the fundamentals behind how people think. As opposed to the popular notion that we think only in our brain, they presented a bunch of examples that showed how our bodies also play a vital role in thinking about something. This concept is called embodied cognition. One of the examples they presented were one which showed how majority of the people tended to lean forward when they were asked to think about their future life; while when the question was flipped by asking them to think about their past, their bodies tended to lean back.


 
2. History of Immersive Experiences
   We were also exposed to various projects from the past that explored experiences that were in the intersection of the physical and virtual worlds. Like, the Sword of Damocles which is considered to be the first VR headmounted display developed in 1968 by Ivan Sutherland at MIT:
   
   
   
   
   Next was the Sensorium developed by Morton Heilig in 1957, it was a simulator for one to four people that provides the illusion of reality using a 3-D motion picture with smell, stereo sound, vibrations of the seat, and wind in the hair to create the illusion:
   
   
   
   
   And finally, no example is complete without Ivan Sutherland’s famous Light Pen demo, which shows the world’s first sketchpad that was operated with a stylus called the light pen:
   
   
   
   
3. Mixed Reality
   Our class was also focussed towards mixed reality and the intersection between the real world and the virtual. We also talked about the different types of reality seen today: IRL->AR->AV->VR.
   
   
   
   
   
   Some of the examples that were presented to us in this space were also quite inspiring, such as:
   
   
   The Meta Cookie: A project which shows that it is possible to change the perceived taste of a cookie by overlaying visual and olfactory information onto a real cookie by using augmented reality:
   
   
   
   
   Sleep no more: Sleep No More tells Shakespeare's classic tragedy Macbeth through a darkly cinematic lens, offering an audience experience unlike anything else. Audiences move freely through the epic story of Macbeth, creating their own journeys through a film noir world.
   
   
   
   
   
   Fragments: Fragments is an experience based on the holo lens that tells a thrilling crime story using your home as the setting. Characters sit on your furniture and talk to you, and you may even find blood stains from the story on your walls.
   
   
   
   
4. Types of immersion The next few lectures that followed from the class talked about the different kinds of immersion that can be created through an experience. Some of them are:
   
   
   Motoric Immersion (Tactical): Sensory-Motoric immersion is experienced when performing tactile operations that involve skill. Players feel "in the zone" while perfecting actions that result in success.
   
   
   Cognitive Immersion (Strategic): Cognitive immersion is more cerebral, and is associated with mental challenge. Chess players experience strategic immersion when choosing a correct solution among a broad array of possibilities.
   
   
   Emotional Immersion (Narrative): Emotional immersion occurs when players become invested in a story, and is similar to what is experienced while reading a book or watching a movie. It is basically empathy.
   
   
   Spatial Immersion (Presence): Spatial immersion occurs when a player feels the simulated world is perceptually convincing. The player feels that he or she is really "there" and that a simulated world looks and feels "real".
   
   
   Moreover, we also learned about taking advantage of the way in which our body works in order to trick our senses and create a virtual experience. The meta cookie shown above is an example of this. Another example is redirected walking, where we can trick our minds to walk along a straight path in a virtual environment when actually we are actually walking along a circular path:
   
   
   
   
   The next example which Josh and James talked about from their experience with designing the holo lens interface was pseudo haptics. Pseudo haptics is an illusion of creating a feedback for an action by using our own body. For example in the holo lens interface you can click on something by simply selecting what you want and tapping your index finger and thumb together, on doing this we get a natural tangible feedback from our own body from the touching of the fingers and it simultaneously serves as the haptic feedback for selecting an action in the holo lens interface.
   
   
   
   
5. Types of prototypes
   Soon after we were exposed to all the basics types of immersions, the teachers talked about the different kinds of prototypes and the techniques that can be used for simulating an experience. Prototypes are very helpful in terms of communicating a concept to others, quoting from the class: “If a picture is worth a thousand words, then a prototype is worth a thousand meetings”. Some of the different types of prototypes are:
   
   
   Aesthetic prototypes: These prototypes are used to communicate how the final object or experience will look like, they don’t necessarily have to be functioning.
   
   
   Functional prototypes: These prototypes are able to simulate the final experience for the user through wizard of oz techniques or in other words through faking the actual technology by using work around alternatives. For example using a sketchpad and notes to represent an ipad or its interface.
   
   
   Technical prototypes: These prototypes are capable of showcasing the fully functional technology required for deploying the project, but it doesn’t necessarily have to look like the final finished version.
   
   
   Moreover, they also talked about techniques like bodystorming, drawing, storyboarding and video prototyping for building and communicating prototypes.

Apart from all the above, we were also given basic introductions to platforms like Unity, Unreal, Aframe (for web based VR), webVR, reactVR (develop VR apps using javascript) that could potentially be used for prototyping.

Wander: Can Yanardag, Keyur Jain, Arvind Sanjeev

As the lectures were happening at the beginning of the week, the same time we were also asked to form teams and start to think about a site, a location at which we want our experience to be set up. And also about the problems we are trying to address at that space. We started off by picking a hostel lounge as our site and then presented this along with the problems we want to address to the rest of the class. Soon after this, we also built a low fidelity prototype of this experience and presented it to the class. Our project is called Wander:

Wander is an interactive surface that allows people staying in shared spaces to collaborate and engage with each other. It provides an excuse for people to communicate in settings like hostels. Wander is capable of identifying people currently staying in the hostel and the popular tourist attractions they have visited in the city through the app the hostel provides.
You can use Wander by placing your keys on the table, after which it detects who you are. Soon after, it asks you to select the landmarks that you wish to visit in the city, after dragging the corresponding tokens into the surface, it shows you the people who have already been there and the people who are currently planning to go there. Through this information, you can ask for feedback from the people who have visited those places and also choose to join other travellers to explore that destination.

The prototype that was developed to explain the concept works through a custom interface that was mapped to a table via a keynote application.

   

Some more experimental projects from the class:

The Other Side: Anoushka Garg, Bora Kim, James Zhou, Stephanie Lee

 

  

Bird's Nest: Esther Bretschneider, Somayeh Ranjbar, Vytautas Gudaitis

  

HoloCubes: Jens Obel, Matt Visco, Benedict Huebener

  

More projects from this class can be seen here: Immersive Experiences.

This is just a short intro to my life at CIID, I will be publishing more posts on the different topics we explored as well as the projects we did through CIID Chronicles. Furthermore, all the credits to the media content for this blog goes to the IDP class of 2017.