All the ARKit apps that I use are on my iPad 9.7-inch (non-Pro model). I love playing AR games on my iPad because of its large display. That being said, I am disappointed with the camera quality. In this post, I want to share my thoughts about the iPad 9.7 rear-facing camera’s technical aspects in association with my personal AR experience.
The distance of the camera from the scene
When I started using ARKit apps I’ve experienced two main inconveniences. The first one is that when you use an app, you almost always put the devices around 30-40 cm (11.8- 19.9 inches) away from your body, even more. This means that if I sit and play an ARKit game near a table, I actually see the game in very a closer range compared to how I see it with my own eyes because the camera is closer to it. This means that when using an ARKit app, you also need to consider that distance. Some of the games that I played were really uncomfortable to play because the game requires more distance to play properly.
My living room is very small and I had trouble playing some game that even don’t require a lot of space, but because the way I hold the device, I get to see a zoomed image of the AR scene. What I do naturally is get back to the same distance to get a large field of view (FOV). This is one of the reasons why we AR glasses are a great idea.
Now, this issue is not just for the iPad, but for the iPhone as well, The thing is that if the lens had a wider field of view, a wider angle lens, this might be less of an issue. I think that the iPad lens focal length is 28mm or 29mm (35mm equivalent). To get a wider field of view we need a lower focal length number, I assume something like 20mm-23mm will be optimal. The thing is that when you use a wider focal length, there are also distortions and I don’t know how optimal ARKit is without that type of lens and whether the optical distortions can be corrected before it reaches the ARKit algorithm for surface detection.
Slow Lens & Small Pixel Size
The second issue, which I think it’s more prominent than the first one is that the iPad’s camera performs really poorly in low-light conditions. According to the official specs page on apple.com, the iPad 9.7-inch has a f/2.4 aperture lens for its rear camera. According to phonearena.com, its 8MP sensor has 1.1-micron pixels.
The back-illuminated technology improves the low-light performance making the sensor more sensitive to light, but still, the light gathering capabilities of this sensor are low.
This is why I always see noise when playing indoors. Even outdoor under bright daylight I can still notice image noise, but it’s less prominent compared to an indoor environment.
The iPhone X, iPhone 8 and iPhone 8 Plus have much better rear cameras. All of those phones have a f/1.8 aperture lens and 1.22-micron pixels. The difference between f/2.4 and f/1.8 is 0.83 stops, almost a whole stop. This means that the iPhone X and iPhone 8 rear-facing camera collects 1.78 times more light than the iPad 9.7 camera. With larger sensor pixels, more advanced sensor technology, and probably better optics, the iPhone will perform much better in low light.
As you already know, ARKit requires good lighting for accurate and consistent surface detection. Without it, it will either won’t be able to detect the surface or it will take more time or you will have a lot of jittering. So some apps that don’t work well indoors due to poor lighting, might work well with the iPhone X and iPhone 8. After all, ARKit analyzes the image from the camera, it doesn’t come with its own proprietary depth-sensing sensor to analyze the real-world scene. If this was the case, the standard camera wouldn’t be an issue because the ARKit will sample the data from the more sensitive depth-sensing camera.
So although I really enjoy ARKit on my iPad, there is certainly room for improvement to make the camera more optimal for a better AR experience. I do hope that the next iPad will come with a more advanced sensor and optics so the overall AR performance could be improved and so the AR experience as well.
What are your thoughts and experience with iPad with ARKit?