Designing Great Mixed Reality Games Part 1

Designing Great Mixed Reality Games – Part 1: Making Great First Impressions

I’ve just finished reading those three sections on the Magic Leap Creator portal: “Core Mixed Reality Principles” and “MR Interaction Goals”, and “New Experience Consideration for MR”. This gave me a good initial understanding of some of the mixed reality design principles and considerations when developing different types of apps for the Magic Leap One headset.

This is my first guide that accumulates some of the official recommendations that I’ve received after reading those articles, alongside my personal recommendations and insights. This is the first part of a set of guides that I intend to write and share with my blog readers. It is aimed at game designers and developers alike. As always, I highly recommend reading the official guides and tutorials for the respective MR platform or platforms you are developing your game for. These set of guides are tailored towards the Magic Leap One headset and its spatial computing platform. This guide is designed to expand over the official guides and not replace them.  If you find any inaccuracy or mistake, please let me know and I will correct them. I hope you find it interesting and useful.

Overview

In this article, I want to share my own recommendations for mixed reality development with a focus on gaming. I know you’ve probably read that some say that the Magic Leap One isn’t mean for gaming and it is aimed mostly to satisfy business/enterprise needs. The device is still a device with all its capabilities. What you do with it is your decision. Aside from the device not being fully tested for outdoor use, there are no very strict limitations to what you can create with it. A game is an app like any other app, which involves interaction with a user interface to generate visual feedback.

If I had to worry about something it would probably be the high price of the unit, which limits its reach. I assume many developers will opt for developing apps for enterprises, as they can charge a higher price for their app and have a larger market to work with. Having said that. we can’t forget the Early Adopters which can be a “good enough” market segment to work with when making your first entry into this market. Building strong roots for this audience can give you a good standing once the Early Majority steps in.

Virtual Reality was not that much different when it first emerged into the market, with the first headsets and the PC requirement started at around $1600 and with games prices relatively high until the market share reaches a point where developers could afford lowering their game prices down and continue to grow organically rather than being required to put a high price tag on their games in order to maintain revenue and survive this tough period.

Targeting the business market might also require continuous support, more expedited bug fixing and other efforts in order to satisfy the needs of businesses. You can always prototype and just showcase your ideas. if you want to go crazy with ideas and really make a strong impression, gaming is a great category to do so, especially considering that most users will use the Magic Leap One indoors.

The Magic Leap One is an extraordinary device like no other. It opens a whole new world of creative possibilities across all verticals, including video games among other. This is a time where you can build a name for yourself or your brand, create a game that will blow people’s minds. I won’t get into the marketing stuff because this is not my specialty. What I can do is to give you some recommendations on how to build great mixed reality games.

Like previous guides that I’ve written about AR (some of my recommendations can apply to AR as well), I will start with some more general stuff and get into more detailed and informative recommendation in future parts.

In this guide, we’ll talk about how to make great first impressions when bringing your virtual content to life when the user first launches your game and how to make sure that content doesn’t go unnoticed.

Make a Fancy Entry

One of the great moments for me is when I launch a new game. That moment where I know I am going to that other place. In Mixed Reality, the user consumed the experience in his own environment, because of that, the weight that the virtual entity is given is much higher compared to standard games.

The user doesn’t expect the environment to change a lot, after all, this is not a VR game. This is why whatever entity you present, especially if it’s an interactive character, you should bring it to life in an expressive, creative and exciting way.

For the user, it will be like opening a gift box. A moment that happens in your own environment, but the main focus is on that unknown new thing that exists inside it. Once users launch your app, they have very high expectations. They want to be surprised, excited and satisfied with that gift you gave them.

You remember that moment in the movie A.I. Artificial Intelligence from 2001. The moment where the robotic boy arrived at the house as a replacement for their son, Martin, who has bee placed in suspended animation until he can be cured of a rare disease. So that type of excitement should be in your game when the user first launcher it. It doesn’t necessarily need to be virtual living being but that entry moment should be a memorable one.

Here are some examples:

  • Make a character make noises and appear slowly behind one of the user’s furniture
  • Make spaceship appear from the sky (top of the room) and slowly land on a previously scanned surface
  • Make a character fly around and then center itself at where the user is looking at (based on the headpose vector), introducing itself to the user a second later
  • Make a character morph out from out of the surfaces in front of the user
  • Make some mid-air fluctuations (probably using a semi-transparent filter) with background music and make the virtual entity appear like it enters the room like from a portal, from another place.
  • Use the 3D spatial audio to make a character appear a certain location and let the character sound guide the user there. For example, if it’s a virtual pet simulator, you can make cute noises coming from near the users’ feet and make the character do some cute noises while it looks up at the user’s head, asking for attention.
  • If you are adding environment decorations, make them appear gradually. For example, if it’s a tree, animate it so it appears like a seed growing until it becomes a full tree. Make it slow and gradual, so it appears that the entity comes to life within the user’s space, and not appear like it was always there.
  • Make so the entity might require certain interaction before it fully reveals itself like the virtual entity reacts when it is in focus when the user gets close to it or touch it. Try to maintain that suspension to make it more exciting and that moment more memorable for the user. Remember that moment in the first Harry Potter movie where Ron and Harry see the Dragon egg hatched, well, similar to that.

These are just a few examples, the creative options are endless.

The important thing is that the virtual content will blend gracefully and naturally into the user environment. Even if you are building a complex multi-level game, it’s important to start out small and gradually adding new content and interactions as the user progresses. When the user first starts the app, don’t overload the user and make him or her concentrate on many things at the same time.

When you present your virtual content in the real world, you can make that content obey Earth’s physics or not. The great thing is that you can do whatever you want with it, but maybe the first entry should be something that is more recognizable and doesn’t scare the user, so retaining real-world physics is something that can be beneficial at starts. For example, you don’t want the user to put the game board on a table, only to find it floating in mid-air or throwing a balloon and seeing it fall like a brick.

Having said that, when the physics don’t follow the same rules as we know it, it does invite experimentation. It’s like being in a chemistry class and discovering some great new chemical reactions or being in a physics class and getting excited by a new physics experiments of objects behaving not as you expected them to be. Try to make the intro part interesting, simplistic, engaging and visually appealing and try to avoid cognitive overload by keeping interaction simple and familiar. Make the more complex interaction saved for a later stage of the app. Failing to do so, might lead to the user exiting the app before even starting to enjoy the great feature you game has to offer.

Initial Placement and Getting the User’s Attention

In a Mixed Reality game, you have the option to place virtual content not just on a surface, but anywhere around the user on constructed mesh surfaces. The boundaries of the experience are completely different than what users are used to in regular games. You can use that to your advantage of course but it’s important to make sure that the user won’t need to spend a long time searching for it.

Magic Leap actually talk about this subject in their documentation. One of the recommendations is to allow the user to choose the boundaries and location in which the app is going to take place. This is similar to what we saw in many AR games, where you first scan a surface, choose a specific place to put the game zone and then resize and relocate it if necessary.

In Mixed Reality, developers might make an intro that encourages the user to look around and scan the environment in the background or use pre-scan before the game starts. Now, we are not just talking about surface like in current AR games. There are plenty of places where you can put virtual content on. Where to place it is up to you and of course based on the particular game you are developing.

To make it less annoying as possible, it’s better if the content appears at a location based on the headpose (maybe in a combination of eye gaze as well, for better accuracy). So when the user clicks “Start” or when the user expects the game to begin, you place the content preferably in the foveated region. The foveated region is only about 18 degrees of your FOV but it’s the region of central focus.

You can create a visual or/and audio cue to get the user’s attention to a certain place in the environment within his field of view. Make it animated because the part outside the fovea is highly attuned for movement. So if the user at a certain moment focuses on another area, you can still get the user attention towards that cue.

You can also make that cue relatively large, but make sure its movement is predictable (if you applied movement to it), make sure it has a good contract against different type of backgrounds (maybe make it glow) and add some location hints in case the content is outside of the user’s field of view (FoV).

Speaking about guiding user attention. Magic Leap gave some recommendations on that matter in their “New Experience Considerations for MR” section, of getting the user to look at where you want him or her to look at. This includes moving the focus object towards the targeted location, creating a trail for the user to follow or use a fast-moving object or effect in the direction of the new location. You can combine it with alarming sound to enhance over the visual cue.

Having a user attention guiding element is a good option to have for certain types of games where the location of the virtual content might be located in areas outside the user’s field of view or outside his main area of focus. The can obviously disable it and activate this feature only if needed.

I remember that one of the things that really annoyed me in AR was when I needed to search for the virtual content or when I realized that there was a main menu in the game but it was above my head the whole time.

The headpose is your best friend here. Magic Leap One C API exposes two Head Tracking modes: Full 6 degrees of freedom tracking for position and orientation (MLHeadTrackingMode_6DOF) and Limited 3 degrees of freedom tracking for orientation only (MLHeadTrackingMode_3DOF). Once you have created a tracker (MLHeadTrackingCreate) you can obtain data from it. The same works for Eye Tracking.

If you are using Unity, you have access to the MLHeadTrackingState struct that contains information about the current state of the Head Tracking System. You can then check the MLHeadTrackingMode to see what tracking mode is being used. I recommend reading the official documentation to learn more about the available classes available for the platform you are using to develop your game.

One way you can use to make sure that some content is visible to the user is “pinning” it within the user’s viewport and make the user interact with it (at least initially) with the Magic Leap One Control. You can find more information about it in the “Interacting with Head Locked Content Unity Edition” section in the Creator portal. This might be good for certain UI elements but it’s recommended to avoid using head-locked content in your mixed reality apps because it is uncomfortable, breaks the immersions and doesn’t feel like a natural part f the world.

Use Mesh Occlusion and Collision

One way that you can make great first impressions is by taking advantage of occlusion and collision against the scanned mesh environment. These are the two central features that make Mixed Reality such an amazing technology. Without it, your game will feel like a standard AR app.

The user’s environment is your playground so try to think outside the box and make use of the environment occlusion and collision to make an impressive start for your game. Something that will wow the user and makes him want to further experiment with what lies ahead.

For example, instead of just having a dragon appear out of thin air. Make it appear like it is coming behind a hidden area (e.g. behind a couch) as it approaches the user and greets him, then make it land on one of the areas in front of the user. By doing so, you’ve just used both occlusion and collision with the scanned environment and this will make very good impressions for sure.

Visual Introductory Elements

The last thing that I want to talk about in this guide is tutorials In many AR games that I’ve tried, I was introduced with popup screens and lots of text that explained how to set up the game and get started.

In Mixed Reality (although many of what I say applies to AR as well), you should avoid any popups or lots of reading as this breaks the immersion. Try to think of it like it wasn’t a technology like you aren’t actually wearing MR glasses but being without them. Imagine yourself just walking in your room and suddenly a virtual popup window appears in front of your face – this isn’t fun isn’t it.

When you build an introductory element in your app, whether in the beginning or throughout the game, make sure it blends seamlessly with the rest of the visual experience. For example, instead of writing: “Move to that location”, add an instructing voice and/or a visual cue instructing the user to move to a certain location. If you add text, make it appear as part of the environment (there are many other things to consider when using text, I will talk about that in future guides).

The goal is to free your mind from many previous game design principles that you’ve adopted from standard screen-based games and adopt new design principles that can provide users with the most immersive mixed reality experience.

Game main menus are probably still here to stay. In some ways I see it similar to virtual reality and augmented reality as well, There are ways to place the menus in different places, but I found standard eye-level menus making text content more readable as you don’t need to look at it in a certain angle, there is no need to search for it because it’s right there in front of you, and in general, you only spend a fraction of a time interacting with it anyway, usually before you start the game.

In-game menus are something different and I there are ways to utilize them in a non-intrusive way so they won’t impair the immersion but I save this topic for later.

Keep in mind that eye-level standard menus can be replaced with simple 3D interactive objects in the real world space, especially if you just have a few options available (e.g. Start, Global leaderboard). If so, consider dumping the standard UI and replace it with one that integrates more seamlessly with the real world scene. Again, the UI is a whole new topic, and I intend to write an in-depth guide about that later on.

Summary

It’s important to keep in mind that the decision that you make should be based on the type of game you are developing, whether it’s an orientation-only game, one designed for a seated position, standing position, room-scale or world scale. Some rules are meant to be broken and you find what fits for your game from trial and error. I’ve seen it before in augmented reality and I was always surprised by the creative ways that developers find to tackle different design issues in their games.

I hope you find this guide interesting and useful. I am spending hours each day learning about the Magic Leap One, diving into the API to get a better understanding of the available features and what they offer. I share any interesting and useful information and insights so you can use it to your advantage when developing future games for this mixed reality platform or other ones to come.

If you are developing for augmented reality, I recommend checking out my AR Game Design guides. You can start with the first one here.