Today I’ve downloaded an app called ‘Just a Line’. It’s an app that allows you to draw white lines in the 3D space, either alone or collaboratively with friends. IT is built using ARCore Cloud Anchors, and because of that, it supports shared experiences across iOS and Android devices. This means that one user can draw with his ARCore-enabled Android device and his friend can join in and share the same drawing space with his or her iOS device.
Collaborative Drawing Experience
When collaborating, what happens behind the scene is that the ARCore device sends a visual mapping from the device to Google servers, where it is processed into a sparse point map where it can be shared and matches against a second device, determine the same position and orientation in relation to the environment, so the drawings can be drawn in the exact same place on both devices. This is possible as both users share the same physical space with a similar mapping description that resolves to the same anchors.
This means that if User 1 draws something in space, it is positioned relative to that cloud anchor, and the other user will see that drawing in the same location and orientation where it was drawn, relative to that same shared cloud anchor. Even if the if the anchor’s position is changed, the objects still maintain their relative position and orientation to that anchor. This is why shared experienced can be very accurate, allowing both sides to observe the object locations in the same location and orientation relative to that shared anchor.
To make collaborative drawing work between two different platforms (iOS and Android), you’ll need ARcore and ARKit enabled devices, with the iOS device running iOS 11.3 or later OS version.
I did try to use the collaborative feature using my OnePlus 6 ARCore-enabled device and my ARKit-enabled 9.7″ iPad, but without success. Both devices recognized each other(“Partner found” on both devices), but the two weren’t able to connect. I point both the Android and iOS camera to the same exact location, both right next to each other but it failed every single time. I tried due to the different camera focal length differences to position the iPad a bit further away so the field of view matches more or less, but it didn’t help. I tried it like 20 times and was disappointed that it didn’t work.
AR Drawing Experience
After seeing that I won’t be able to make the collaborative feature work, which was one of the reasons I wanted to test this app in the first place, I went on trying it without the shared experience feature.
The drawing experience isn’t great, especially when you do it alone. This is an experimental app, so don’t expect to find Paintbrush like feature here, although this app is made by Google Creative Lab
By the way, this is an open source project which you can download from GitHub and try it yourself, modify it to your liking.
I did experience some drifting, which happens more prominently indoor whether outdoors it worked more accurately.
There point map isn’t saved to the cloud, which means there is no persistence. I tried closing the app, relaunch it and see whether the app will redraw the scene in the exact place but I just started with a plain canvas.
Furthermore, I’ve noticed that when I move to a certain place, suddenly a drawing that I painted like 3 meters away, appeared right in front of me. when I moved to the original location where I’ve drawn the shape, it suddenly snapped back into place. It was a weird experience. I guess when the app can’t recognize the scene, it might put the drawing in front of the user as a default but again, I am not sure about that.
The lines are drawn where the camera is located. So when you tap and hold on the screen, a line is drawn in that place. In terms of accuracy, the drawing was accurate. The line is drawn a few inches away from the device so you won’t need to bump into something if you want to make accurate drawing near objects in the real world.
Overall, the drawing experience felt good, but we can’t ignore how limited it is when you have to do it with your device and not having your hands free to do so.
The phone is your window to the word, and it isn’t a good experience when it is used as a paintbrush and as a viewer. This is why, in my opinion, this type of drawing experience isn’t designed well. It can be great with AR glasses like the Magic Leap one, but I found it quite annoying to use using it that way.
Now, I am not saying that all drawing apps are bad in augmented reality, not at all. It’s just this drawing experience wasn’t great, but again, it’s an experimental project that was designed to showcase ARCore’s shared experienced more than anything else, I guess.
Take the app Blue Sky Paint for example. this is a fantastic drawing app. It also supports collaborative drawing experience, but here you don’t draw in the 3D space but use the sky as the canvas. It uses sky occlusion and because that canvas is huge, you can just draw with your finger while the device is pointed at the same location in space. Even if you are drawing on a larger scale, you’ll probably only move the device just a bit and this will help you cover an even much larger portion of the skyline area in front of you. Blue Sky Paint is a drawing AR app done right.
One of the reasons I bought the OnePlus 6 is to be able to try those great shared experienced between Android and iOS. Well, it didn’t work well for me using “Just a Line” and I will need to search for other apps that use Cloud Anchors to get a first-hand experience of how it actually works.
I do hope that maybe some developer can find the reason why it didn’t work for me and I will be able to test it out.