online multiplayer AR and accessibility

Online Multiplayer in Augmented Reality & Accessibility

I love playing online multiplayer games, I mostly play them on the PlayStation 4, Nintendo Switch or on mAndroid phone or iPad. when it comes to augmented reality, most of the games that did support multiplayer only supported local multiplayer and not online multiplayer.

I Love Playing Online Multiplayer Games

Of course, having a local multiplayer option is great, and I was really excited when I’ve heard that that feature is available for ARCore and ARKit. Having said that, I am an adult gamer, and I don’t have many local friends whom I can play with. Even if I had many, sometimes I want to enjoy a multiplayer gameplay experience without relying on other people to play it. I enjoy the social aspect of online games and many of the games that I do play (no-AR) are online multiplayer games.

I Want Online Multiplayer in Augmented Reality As Well

So I raised a question about how can we deliver an online gameplay experience for augmented reality games. I am referring to games, but this can be any multi-user AR/MR experience.

Playing AR games is in many ways different than playing a game on your PC, phone or tablet and it’s also different than playing games in Virtual Reality (VR). You can clearly see that most of the most popular games in the world are online multiplayer games or at least have a multiplayer game mode available, like: Fortnite, Grand Theft Auto V, Tom Clancy’s Rainbow Six Seige, Overwatch, League of Legends, Dota 2, PlayerUnknown’s Battlegrounds, Call of Duty: Black Ops IV, Rocket Leauge, Roblox, Counter-Strike: Global Offensive—well, you get the idea.

We Are No Longer Confined to a Screen – We are Free!

With Augmented Reality & Mixed Reality, whether these are played using a mobile phone or a Mixed Reality headset—can benefit a lot when being played while moving in the environment, both indoors and outdoors, rather than resembling the desktop/console gaming approach where you sit down in front of a screen or like VR when you might be standing but active in a relatively confined game area.

Space is the canvas onto which developer are building their experience. They are no longer confined to a limited square screen space. Magic Leap One, for example, can scan your entire room and developers can design experience that takes place on that entire space. In the near future, this will move outdoors with the ability to mesh real physical objects further away from the camera and with AR cloud, the physical structure of our outdoor and indoor environment can persist and have itself virtually/digitally replicated online. This means that the physical world (e.g. buildings, roads, statues, etc) and it’s content can be part of the mixed reality experience.

Now if you add AI (machine learning & deep learning) and computer vision into the mix, alongside real-time detection of moving objects, everything around you can be used to create unique and compelling experiences. We are no longer confined to a screen or to limited physical space – We are free!

With AR/MR we can have games use the environment and display virtual content that interacts with physical objects and in context as an object out there will eventually be recognizable in a high degree of accuracy. Machines will be able to learn and even understand what an object is even if it is seen for the very first time or at least give a close prediction and even do that better than we can.

So How We Do That?

There are several ways to bring online multiplayer experience to AR/MR videogame applications. In the next video, I’ll talk about a few of them.

As you can hear and see in this “video, there are ways to deliver an online gameplay experience with augmented reality / mixed reality. Whether these experiences are being consumed using a mobile device like a phone or a tablet or using mixed reality glasses like the Magic Leap One.

The idea is to have cour experience also support cross-platform, works across different technologies (e.g. AR & VR & Desktop) in order to increase the player base and allow players to play the game if they can’t play it the way it was meant to be played originally with just using AR glasses or using standard mobile AR, or if for any reason (e.g. illness, disability, time limitation, weather limitation, etc).

You can decide to create a game that works across all reality technologies (AR/MR/VR) or add more like desktop, console, etc.

The Importance of Accessibility

There is another important topic that I wanted to talk about, that is derived from what I said above and it is related to accessibility. By this I mean, allow people with disabilities to be able to take part in the experience like those who don’t have these type of disabilities.

Imagine a person who is confined to a wheelchair and can’t walk and a mixed reality game is taking place outdoors that requires a lot of navigation through the streets. This is not the type of experience that might be accessible for a large portion of handicapped people.

Imagine a person who is visually-impair yet this person still wants to be able to take part in the experience. Now I am not talking about color-blind, but people who literally can’t see.

Let’s look at an example not related to AR/VR. Let’s take the popular game Fortnite by Epic Games as an example. Imagine a person who heard about Fortnite, this person is blind. For obvious reasons, playing a game in any of the available options is completely inaccessible to that person. If we could provide a way, unique interface design for blind people to take part in the game and have an effect on it, it would be amazing.

This, of course, need to be designed carefully because the controls need to be designed while understanding a blind person limitations. I am not an expert in that field, but I can give a rough example. Imagine the blind persos, let’s call him Josh. Josh can control a flying drone with a joystick. His goal is to fly (or move on the ground if it’s a small ground vehicle) and help this squad to detect enemies. The drone has very good ability to detect sounds and distinguish between sounds of his teammates and the enemy. Josh uses the joystick while listening to the sound in the environment. If a voice is detected, Josh can send a signal to his teammates of the location of the enemy. The enemy, on the other hand, can shoot the drone down. Josh needs to be careful so the drone won’t get shot. He can hear the sounds of the shots and can maneuver the drone around. If the drone intersects with a building or something haptic feedback and sound is heard.

We can also add some features of allowing for an autopilot for a different location or if the Josh wants to team up with his squad with voice recognition, he can say a command like “Join team” and the drone will automatically fly there.

This way, Josh can play with his friends and enjoy playing a game and socialize with other players. This is just a simple idea, of course, you can make something of your own and there are unlimited ways to create unique and compelling gameplay experiences that can allow both disabled people enjoy playing games with those who are not disabled.

This is something that I want to see now and in the future. I want developers to unite all gamers, regardless of their disabilities and allow everyone to participate in an online game.

There are of course accessories that allow people with disabilities to play standard games, which is great, but we need to think of ways where we can offer people with disabilities a way to play a game without buying any additional accessory if possible, or if it is required, have this accessory available so they can play a popular title with their friends and feel part of the excitement and not feel that they are left out because of their disabilities.

In future articles I will bring many more ideas about how this can be implemented but I need to spend more time reading about this accessibility topic so I can provide you with accurate data, statistic and ways to implement it for different types of games, with emphasize on AR/MR games, but also for standard games as well.

If you solve some of this probably regarded accessibility, you already improve the usability of the app altogether and allow people that don’t necessarily have any disabilities to enjoy the gameplay experience when they can attend the AR/MR game in the original way it was meant to be played. More on this topic in a separate article.

Conclusion

Maybe a conclusion isn’t the right word because I still yet a lot that I want to talk about, and I will do this in future articles.

I want questions to be raised, ideas to be discussed, people better understanding that other person who want to take part of a great game but can’t, especially in times where many MR experiences will be taken place outdoors. It mmight not be that hard as you might think to implement features that can allow people with disabilities take an active part of the gameplay experience.

I did want to open this topic because I already see part of that picture of developers making great mixed reality applications for AR glasses and mobile AR and I I want to make sure that these topic are mentioned and discussed so we can create better gameplay experiences that more people can enjoy and take part of.

If you enjoy this article and find it interesting and useful, please help spread the word among fellow developers. Thanks so much for reading.