Michael Swanson shared a glimpse into Magic Leap One Wayfair.com home furnishing and home decor app for the Magic Leap One. This app seems to work similar to its mobile augmented reality variant. Although from first glance, it seems less feature rich.
The interesting part that I wanted to talk about is the occlusion and meshing with this app compared to its augmented reality equivalent. I don’t have the Magic Leap One so I didn’t get a chance to try it out. However, I did download the Wayfair iOS app and installed it on my iPad, so I can get an idea of how it works and have a better understanding of the differences between the two.
I’ve tried quite a few home decor AR apps in the past, including Housecraft, Sayduck, Room Remix, IKEA Place and many other similar apps.
When you use an augmented reality app, you can place furniture or home decoration objects on a detected surface or if the app allows it, on an extension of a scanned surface. So even if you don’t scan the entire room, you can still place a couch for example, on a floor outside the scanned area.
In Wayfair iOS app, you have two options to preview the furniture or home decor object. either as an image overlay or in 3D in augmented reality. If a 3D AR preview is available, you’ll see a “View in Room 3D” button below the image of the item.
The app does a great job placing the virtual item in your room in its real size representation so you get to see almost exactly how it would look like when you buy it.
The main differences are that you need to rescan the surface every time you want to place a new item and there is no occlusion.
What you see above it’s not a bug. It’s a lack of masking of that part that should be hidden behind the cabinet. If I chance and angle and look at it from slightly on the right side, it will blend naturally with the real world environment.
Now let’s take a look at this video by Michael Swanson.
A few things to notice here. First of all, you can see that there isn’t any need for a rescan of the surface every time you place a new item. Second, you can see that Magic Leap One supports occlusion. This feature is in the core of the Mixed Reality framework. When you scan the environment, it starts a room scanning and meshing process that allows the app to be aware of the physical structure of the environment. Those scanned physical entities are converted into a 3D structure. When you put a couch for example behind a table, that part that is behind the table will be masked out.
In AR (without occlusion) the content appears on top of the video stream, the real world scene. And without occlusion, there is no way to mask certain parts of the object so they appear like they are hidden behind, after or below real-world objects.
You can see that even with Magic Leap One, the occlusion effect isn’t perfect and this is due to improper meshing process that for some reason, didn’t scan the edges of the table in great accuracy, so it appears like part of the couch is inside the edges of the table. However, this leads to a much more realistic visualization and better representation of how the object would look like if you indeed buy it and place it in your room.
Leaving the inaccuracies aside, you can see the huge potential of having an occlusion in those type of home decor and furnishing mixed reality apps. I still didn’t like the partially-transparent visuals that make the furniture appear virtual and not realistic.
However, overall, I was pretty damn impressed, to be honest. Mixed Reality is a great technology for these type of home renovation applications and I’m sure it would be used greatly by furnishing, home decor, and home renovation companies.
Apps like the one you see in this video will become better and better as technology evolves. 3D Objects will inherit the real world lighting, and virtual 3D objects that you integrate with the real world will appear more real at a point that you will have a hard time telling what is virtual and what is real.