2014 IEEE Virtual Reality and 3D User Interfaces

The IEEE Virtual Reality and 3D User Interface Symposium are two of my favorite conferences of the year (fortunately they happen to be co-located!). I was able to catch up with many colleagues that I don’t often get to see, and I had the opportunity to meet quite a few new people.  Although I can’t write about the entirety of both conferences, as that would be quite a lengthy article, below are some of the combined highlights along with my personal experiences and perspectives.

VR Tutorial

Talking about virtual reality user interfaces.

Jason Jerald of NextGen Interactions discussing avatars and user interfaces using the Oculus Rift and Razer Hydra.

The week started with a tutorial I organized for VR Development using Unity.  I began the first session with an overview of Unity and the Oculus Unity SDK.  For the second session, I moved onto a discussion of latency, user interfaces, and some tips on reducing simulator sickness.

Here is a video of the second session:
IEEE VR Tutorial Session 2.

Lecturing with a Rift

Jason practicing his speaking skills while wearing the Oculus Rift.

Jason discussing his experience working with Virtuix for the game shown on ABC’s Shark Tank.

Jason discussing his experience working with Virtuix for the game shown on ABC’s Shark Tank.

Sebastien Kuntz, Founder and President of of I’m in VR, then provided an overview of his Unity MiddleVR software. He started off with the basics and then allowed the audience to follow along and start integrating components together in a more detailed format.

Arno Hartholt, lead of the Virtual Human Toolkit from the University of Southern California Institute for Creative Technologies, and Lakulish Antani of Impulsonic also showed off their work within Unity. I really wanted to see these sessions but was unable to since I had to attend the Workshop on Immersive Volumetric Interaction and also present the paper “Bimanual Selection and Interaction with Volumetric Regions of Interest”. I guess that is a sign of a good conference when you can’t possibly see everything you want to see!

We ended the tutorial with the audience asking the panel speakers questions.

VR Keynote

Henry Fuchs discussing the Facebook Aquisition of Oculus VR.

Henry Fuchs discussing the Facebook aquisition of Oculus VR.

Henry Fuchs gave an inspiring keynote that was one of the best talks I have ever witnessed!  Henry started by saying that he changed his entire speech after arriving to the conference and discussing the acquisition of Oculus VR by Facebook with other colleagues.  He believes that this is the best thing that could have ever happened in VR as it validates the work many of us have been doing for many years, and the industry will be encouraged to expand into VR and innovate.  He ended by stating that not many professionals have a chance to change the world over the course of their entire career, but in our case the timing is right for us to have the rare opportunity to change the world.

Henry Fuch’s summary and concluding thoughts about the current state and future of VR.

Henry Fuch’s summary and concluding thoughts about the current state and future of VR.

HMD Panel

IEEE Virtual Reality panel on head-mounted displays

The HMD Panel left to right—Jason Jerald—Founder and President of of NextGen Interactions, Stephen R. Ellis—Research Scientist of the Advanced Displays Group at NASA Ames Research Center, Yuval Boger—CEO of Sensics, David A. Smith—Chief Innovation Officer at Lockheed Martin MTLS.

I led a panel of HMD experts consisting of Steve Ellis–Research Scientist of the Advanced Displays Group at NASA Ames Research Center, Yuval Boger—CEO of Sensics, and David A. Smith—Chief Innovation Officer at Lockheed Martin MTLS.  The four of us collectively had more than 50 years of experience with virtual reality!
Steve started off the panel with a brief history of HMDs. I think his slide showing some HMDs over the last 400 years might be my favorite slide of all time!

Various HMDs since 1613.

Various HMDs since Galileo’s first in 1613.

Steve continued by talking about what Steve Jobs, if he were still with us today, might consider to be the essential requirements of HMDs.

What Steve Jobs would think is important for HMDs.

What Steve Jobs would think is important for HMDs.

Yuval Boger discussed the different parts of of HMDs and other VR components that will be useful in the future (e.g., eye tracking, biometric sensors, gesture recognition).  I asked a tough question about whether low-cost commodity HMDs would overtake professional high-end HMDs in a similar way that Nvidia put Silicon Graphics out of Business over a decade ago.  Yuval’s great response was that HMDs are different than graphics cards and there would always be a high-end market that would require ruggedized HMDs (e.g., for military and medical applications where HMDs would have to be certified and withstand harsh conditions).

David Smith talked about how we are creating a totally new medium and language in a similar way that film and radio had to be invented .  I could not agree more with David’s claim that great content designed specifically for the experience is what matters—porting existing content to VR will never result in the ideal experience.  David also stated that we need to be better at selling the story and emotion if we wish for mainstream adoption.  Listing a set of features is not going to get us there.

Demos, demos, and more demos!

One of my favorite parts of a conference is trying the demos.  This is the real test.  After all, just because a system sounds great on paper, it many not work the same in practice.

Research Demos
My good friend Luv Kohli ran the research demos this year, and the research demos team did a great job of selecting some really innovative demos.   It was impressive how so many research groups were able to bring their working demos to the conference.  One of my favorite demos was from my friend Evan Suma’s group out of the USC Institute for Creative Technologies where they were scanning in bodies (i.e., attendees) using the Kinect.  They scanned me into their system and into my email—I can’t wait to get put my avatar into a VR environment!

University of Minnesota demos
Some universities are fortunate enough to have a VR research lab.  The University of Minnesota has three research labs/groups that focus on virtual reality!  They have a full CAVE as well as two HMD labs both with large area tracking where we received some cool demos.

3DUI Contest

3DUI Contest Co-Chairs from left to right: Rob Teather from McMaster University, Gerd Bruder from University of Würzburg, Jason Jerald from NextGen Interactions, and Sharif Razzaque from InnerOptic.

3DUI Contest Co-Chairs from left to right: Rob Teather from McMaster University, Gerd Bruder from University of Würzburg, Jason Jerald from NextGen Interactions, and Sharif Razzaque from InnerOptic.

As one of the 3DUI contest co-chairs this year, I was very happy to see some extremely innovative systems participate in the challenging task of annotating hierarchies of point-cloud datasets .  To view the systems in action, head over to http://3dui.org/contestants.  The winning systems that shared first place were:

• “Go’Then’Tag: A 3-D Point Cloud Annotation Technique” by Manuel Veit and Antonio Capobianco from Holo3 at Schiltigheim and ICube at theUniversity of Strasbourg.
• “Slice-n-Swipe: A Free-Hand Gesture User Interface for 3D Point Cloud Annotation” by Felipe Bacim, Mahdi Nabiyouni, & Doug A. Bowman from the Center for Human-Computer Interaction and Department of Computer Science at Virginia Tech.

The French Team: one of two winners for this year’s 3DUI contest “Go-Then-Tag”—congratulations!

The French Team: one of two winners for this year’s 3DUI contest “Go-Then-Tag”—congratulations!

The Virginia Tech team: one of two winners for this years 3DUI contest are “Slice-and-Swipe”—congratulations!

The Virginia Tech team: one of two winners for this years 3DUI contest are “Slice-and-Swipe”—congratulations!

One team won a voucher for a Sixense Stem system (which will be shipped this summer) and the other team won a set of Leap Motion sensors.

Gaming Social
Zach Wendt, President of IGDA Twin Cities, organized a social gaming event Wednesday night.  This was a great chance for VR professionals to meet the local gaming community in a casual environment.  Rob Lindeman of the Worcester Polytechnic Institute and I were happy to demonstrate a couple of VR Games (Rob showed one of his students projects AaaAAaaaAaaCULUS!!!! and I showed a version of my VR Apocalypse game shown on ABC’s Shark Tank with Virtuix ) to several individuals that had not previously experienced VR.

Upcoming VR Events

East Coast Game Conference (April 23-24, Raleigh, North Carolina).
I will be speaking about VR and also showing my VR Apocalypse game on the exhibit floor VR where you can try the game for yourself!

Neurogaming Conference (May 7-8, San Francisco, California).
The potential of integrating neuro sensors with virtual reality is largely unexplored and I expect plenty of attendees and speakers to have ideas on how to mix the two. I’ll be leading the panel “Immersive Experiences—Virtual Reality Neurogaming” with Palmer Luckey—Founder of OculusVR, Ana Maiques—CEO of Neuroelectrics, and Amir Rubin—CEO of Sixense Entertainment.

Silicon Valley Virtual Reality (May 19-20, Mountain View, California)
I will be leading a panel on locomotion and simulator sickness.

Augmented World Expo (May 27-29, San Jose, California)
I will be speaking about virtual humans in augmented and virtual reality.

Posted in blog Tagged with: , , , ,

Virtuix Pitching Virtual Reality on ABC’s Shark Tank using the Oculus Rift and NextGen Interactions’ VR Apocalypse

Shark Tank Virtual Reality Virtuix

Virtuix CEO Jan Goetgeluk pitching to the Shark Tank.

Virtuix CEO Jan Goetgeluk recently pitched his Omni-directional virtual reality treadmill to ABC’s Shark Tank. Jan has been demonstrating his omni-directional treadmill along with the Oculus Rift and the NextGen Interactions’ game VR Apocalypse. The virtual reality gaming system has been presented at various conferences and was recently featured on The Gadget Show (the UK’s most popular tech show). The system allows players to physically move around in virtual reality resulting in a feeling of literally being inside or immersed in the game. If you have an Oculus Rift then you can experience this Shark Tank virtual reality game for free (link below)!

Here is a video of the game VR Apocalypse.

If you would like to receive our newsletter with occasional updates on events, new games and updates, and industry news then
sign up for our newsletter.

You can also follow us on Twitter

If you would like to learn about how you could apply virtual reality and related technologies to your industry then sign up for a
free consultation.

If you have an Oculus Rift and would like to play VR Apocalypse then you can download the game here:
Free virtual reality game VR Apocalypse


Posted in blog, gaming Tagged with: , , ,

VR Gaming Event of the Year! Cloudhead Games, Sixense, Virtuix, Project Holodeck at SIGGRAPH!


I am so honored to be presenting along with the leading pioneers in virtual reality gaming! It is going to be an interesting year–this will be a great chance to get at a peak of what is coming. Everything that will be shown has been designed specifically for virtual reality–expect to see the best virtual reality demos that the world has seen.







Come see the latest innovations in virtual reality! Individuals from various organizations will showcase what they are working on and discuss what the key challenges are for this fascinating medium. Projects to be presented include MakeVR, Zombies on the Holodeck, The Gallery: Six Elements, and the Virtuix Omni. During the second half of the event, the presenters will give you a chance to try their systems for yourself.

MONDAY, JULY 22, 2013 – 9:00AM TO 11:30AM
Anaheim Convention Center
Birds of a Feather
Room 213 B

Posted in blog, Conferences, gaming

Ideas needed for the 2014 3D User Interface Contest

World Builder by Bruce Branit

The motivation for last year’s contest: the short film World Builder by Bruce Branit. This film shows what the future of 3D user interaction might look like.

I am honored to participate on the 2014 3D User Interface Conference Committee as a Co-Chair of the Fifth Annual 3DUI Contest!  See the summary review of the 2013 contest where NextGen Interactions along with Sixense achieved second place for an early version of MakeVR!

This year we are doing something a little different and decided to ask for input on what the challenge should be this year.  Below is a description of what we are seeking.

IEEE 3DUI (3D User Interfaces) will be holding its 5th annual 3DUI Contest in 2014. The contest is open to anyone interested in 3D user interfaces, from researchers to students, enthusiasts, and professionals. We believe the annual contest helps the community find innovative solutions to classic 3DUI problems (e.g., navigation, selection, manipulation) through friendly competition that encourages outside-the-box thinking, unleashing of creativity, and demonstration of new interfaces! The goal of the 3DUI contest is to both challenge and harness the 3DUI research community to solve real-world 3d user interface problems. See http://www.3dui.org/2013/contestants.html for last year’s contest.

This year we are opening up the creation of the contest task by reaching out to our friends across both academia and industry. We need your help to come up with contest ideas! What are some challenges and unsolved problems in 3d user interfaces?

If you know of
– a 3D application that could benefit from a breakthrough user interface,
– a 3D task that would benefit real users, or
– a 3D commercial device that needs a killer application
we’d like to here from you!

We will require contestants to demonstrate their system onsite live, so please keep this in mind when proposing ideas.

Please email us your ideas by midnight July 1st at contest.chairs@3dui.org

And please forward this information on to your colleagues!

Thanks,3DUI 2014 Contest Chairs
Jason Jerald, NextGen Interactions
Rob Teather, York University
Gerd Bruder, University of Würzburg
Sharif Razzaque, InnerOptic


Posted in blog Tagged with:

Robots, Guns, and Explosions in VR with the Oculus Rift and Razer Hydras


Here is a video of my first prototype VR game called TrainWreck. The idea is to put together some different interaction concepts to find out what gamers are most interested in and to get feedback so that I know what to focus on next. I know VR but I am not an expert in games–so any feedback of what you would most like to see is greatly appreciated. Sorry for the end where my screensaver kicked in–that was basically the end of the demo.

I will be showing this at the East Coast Game Conference on April 24-25 in Raleigh, North Carolina. Stop on by if you would like to check it out.

Thanks to Danny Woodall from Sixense for putting together the Tuscany Unity demo as the project setup and source code was largely based on that demo.


Posted in blog Tagged with: , , ,

Review of The Future of Consumer Virtual Reality Panel with Industry Experts

From left to right: Jason Jerald of NextGen Interactions, Palmer Luckey of Oculus VR, Sebastien Kuntz of i'm in VR, David A. Smith of Lockheed Martin GTL, Amir Rubin of Sixense Entertainment.

I put together a panel titled “The Future of Consumer Virtual Reality” at the IEEE Virtual Reality Conference in Orlando, Florida.

I was honored to have industry leaders participate on the panel:
Palmer Luckey – Founder of Oculus VR and creator of the Oculus Rift.
Sebastien Kuntz – Founder and President of i’m in VR and creator of the VR Geeks association. Sebastien focuses on VR middleware and creating cheap VR experiences.
David A. Smith – Chief Innovation Officer and Senior Fellow at Lockheed Martin GTL. David previously founded several companies including Virtus Corporation, Red Storm Entertainment, Timeline Computer Entertainment, Neomar, and Teleplace.
Amir Rubin – Founder and CEO of Sixense Entertainment. Amir previously founded the companies Interactive Light, Transparency Software, 2Train4, and Immersia.

VR Panelists

From left to right: Jason Jerald of NextGen Interactions, Palmer Luckey of Oculus VR, Sebastien Kuntz of i’m in VR, David A. Smith of Lockheed Martin GTL, Amir Rubin of Sixense Entertainment.

Each panelist spoke for a few minutes about the future of VR before answering questions from myself and the audience.

David Smith was the first to present.  David talked about how the mouse changed the way we interact with computers in the 80s even though it was first invented by Engelbart back in the 60s, and shared similarities of how the mouse evolved with the way augmented and virtual realities are evolving today–for example, early users didn’t know what to do with the mouse even though it is hard to imagine today not understanding how the mouse works. David also talked about how such technologies enable us to see the un-seeable and how the computer is a full participant in the conversation.  David has created optics that provide a field of view of 180 degrees! I got to try his display and the experience provided by such optics is going to be nothing short of amazing.

Sebastien Kuntz talked about how consumers will win with VR.  However, hardware must be standardized based on perceptive capabilities. Sebastien also made a great point that even in the worse case that if consumer VR fails, the professional VR market will still grow.  Sebastien thinks the hardware is already good enough but we don’t yet know how to make the most of it, and adapting existing applications is hard. Thus we should start with simple applications since VR is a new medium and we must explore and experiment with the basics first.

Palmer Luckey talked about how consumer VR is not only possible, but that it is inevitable.  We can leverage mass market technologies such as graphics hardware, high-density mobile displays, and high-performance motion trackers–these are all available today at prices consumers can afford.  Palmer stated that a primary motivation for consumer VR is that a long game console cycle and a potentially disappointing refresh has left gamers wanting something new. Major game developers are backing VR tech, although even small audiences motivate indie developers.

Amir talked about the dream of being immersed within one’s art and how that is a motivation for him. Amir remembers talking with the artist David Hockney and how David wished to be immersed in his art and did so by placing his paintings around himself in a room. Amir did not understand the concept at the time but now understands the dream of David Hockney and is working towards that dream of immersive art with MakeVR–an immersive two-handed artistic system that works with Sixense Razer Hydras and the Oculus Rift.

A good panel has some disagreement–if everybody says the same thing thing then then attendees are going to fall asleep or leave. Palmer stated that VR game developers are not creating new immersive experiences for the money but instead are doing it for the experience in itself. Amir strongly counter argued by saying if he told his shareholders what Palmer just said then he would be out of business. His point was that if we want any technology to go mainstream then we need to show large corporations and  investors how the business can be profitable.  Profitability is essential for long term success, otherwise VR will only remain a hobby for a niche market that dabbles in simple experiences.

The panelists discussed challenges of porting existing games and how the best games built specifically for VR will be the best.  Sebastien Kuntz noted that creating experiences across different platforms will require completely new implementations that are built specifically for that technology. This may be obvious when comparing a traditional console experience to a head-mounted display experience, but he noted the experience is different even across HMDs due to different fields of view, different brightnesses, tracking capabilities, etc.

Mark Bolas from USC ICT challenged the panelists to give a number of how many HMDs would be shipped next year. All panelists other than Palmer Luckey (who stated one million units, he also estimate 15000 Oculus Rifts have been ordered) would not give an answer because they felt it is a completely speculative question. David Smith made the point that the market could go either way–we could return to the 90s where VR was overhyped and under delivered resulting in the VR crash of the 90s or things could really take off with millions of units being shipped–it is just too early to tell.

Overall, I greatly enjoyed hearing the speakers talk about their view of the future of consumer VR.  I look forward to moderating my next panel titled “Immersive Experiences: Augmented and Virtual Reality” on May 1-2 at at the Neuro Gaming Conference and Expo in San Francisco.  The panelists will be Palmer Luckey–Founder of Oculus VR, Scallie Laurent–CEO and Founder of Atlantis Cyberspace, and Walter Greenleaf–Director of the Mind at the Stanford Longevity Institute.

Posted in blog, Conferences Tagged with: , ,

The Fourth Annual 3D User Interface Contest

The Fourth Annual 3DUI contest was held on March 17, 2013 at the 3DUI conference in Orlando, Florida.

Nine finalists teams participated in the contest. Congratulations to Jia Wang, Owen Leach, and Robert W. Lindeman from Worcester Polytechnnic Institute for winning the 3DUI contest with their entry DIY World Builder: An Immersive Level-Editing System.

I participated as a member of the Sixense Entertainment team and we took second place! Our system MakeVR is a two-handed modeling interface that allows users to intuitively build complex worlds in 3D space. Because it was designed as an immersive VR system, it is a natural fit with the Oculus Rift and in fact is already fully integrated with the Rift. The system is also fully integrated with a professional CAD engine, so the user-created models can be printed with a 3D printer. Here is a video of our submission.

We are continuing to develop the system and rumor has it that we will be launching this prototype as a Kickstarter in April in order to refine the product and get it into user’s hands (literally).  The most up to date information on the product is available here.

Congratulations to the other nine contest finalists. All video submissions are available at http://www.3dui.org/2013/contestants.html

Posted in Conferences Tagged with: , , , ,




Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.