Material Detection for AR MR Applications

Material Detection Complimentary Technology for Mixed Reality Applications

Today when I saw this video demoing Magic Leap, I thought about what if there was material detection component in Magic Leap One that allows it to detect surface and object’s materials and applied direct physics to meshed polygons based on a material scan. There are already laser camera and different methods (e.g. spectrometric radiography) that do that.

The goal is to be able to give 3D meshed environment scan physical properties based on the real materials of the objects and surfaces that are being scanned. So for example, when you drop a virtual basketball (with real basketball physics) on a water surface, the ball will behave like it was dropped on water. If you drop the ball on your soft bed, that ball won’t bounce up, because the bed will absorb most of its drop kinetic energy.

This is a great complementary technology to both Augmented Reality and Mixed Reality applications. The question is what are the currently available solutions, what are their costs, are those solutions miniaturized and can fit a small wearable device like the Magic Leap One or a mobile phone?

This is a feature that can make a huge difference for any Mixed Reality application, considerably increase immersion and make the virtual content interaction feels much more believable.

I will spend time reading more about this topic this week. I am interested in hearing your thoughts about it.