You may have already read about a time-of-flight sensor and a 3D camera in recent months , including here on iCulture. The iPad Pro 2020 is the first device to have these sensors and the iPhone 12 Pro and the iPhone 12 Pro Max also have such a sensor. Apple calls it the LiDAR Scanner. But what exactly does the LiDAR Scanner do and what can you do with it?
What is LiDAR?
LiDAR stands for Light Detection and Ranging. It is a technique with which a device can scan the environment and thus calculate distances to objects. In self-driving cars, this is often used in combination with other techniques, such as radar.
LiDAR uses small invisible green lasers. By the reflection against an object, the device measures how long it takes for the light to be reflected. Based on this, a calculation is made of the distance to an object. That is why this is also called time-of-flight (ToF): the time with which light ‘flies’ through the air. NASA also uses LiDAR for Mars landings.
On the iPad Pro we find the LiDAR Scanner in the camera hump, to the right of the two lenses. The LiDAR Scanner works together with the cameras, motion sensors and the fast processor to scan the environment, measure distances and recognize objects. The LiDAR Scanner in Apple devices can scan objects up to 5 meters away. Many lasers are sent and received every second, which means a lot of calculations are involved. The A12Z chip handles all those processes.
On the iPhone 12 Pro (Max), the LiDAR Scanner can be recognized by the black dot under the camera. Apple says: “The Pro series has an all-new LiDAR scanner, which can measure light distance and use pixel depth information. This technology delivers faster and more realistic AR experiences and six times faster autofocus in low light, making photos and videos sharper and shorter recording time. ” So it is not only useful for AR (like on the iPad Pro) but also for sharper photos and better image quality.
Which Apple Devices Have a LiDAR Scanner?
These devices are provided with LiDAR Scanner:
- iPhone 12 Pro (2020)
- iPhone 12 Pro Max (2020)
- 11-inch iPad Pro 2020
- 12.9-inch iPad Pro 2020
So it seems that this sensor is mainly reserved for the Pro devices.
What can you do with the LiDAR Scanner? What are the possibilities?
As indicated above, the LiDAR Scanner on the iPhone is mainly used to determine depth more quickly when shooting. This produces sharper photos, especially when shooting under moderate lighting conditions. The photo below is an example, taken by Apple itself.
Snapchat is going to use the LiDAR Scanner to make special lenses. Here is Lens Studio available. The scanner measures the environment so that the Snapchat lens can give a more realistic image of objects and surfaces. By combining the power of the A14 Bionic chip and ARKit , you can display thousands of augmented reality objects in real time.
In the iPad Pro , Apple mainly uses the LiDAR Scanner to improve the augmented reality functions. If you have ever used an AR app , you will recognize that you have to calibrate the app before you can use the augmented reality functions. You then have to move your iPhone or iPad back and forth so that the cameras can scan the environment. Thanks to the LiDAR Scanner, this is not necessary: the environment is recognized immediately.
The device with the LiDAR Scanner is also more aware of the environment. This means that you can, for example, place a virtual chair behind an object in the real world, such as a table. This makes virtual objects blend much better in the real world, resulting in a much more natural appearance. AR objects also stay in place better while you walk around them and take the presence of real people into account. This means that you can walk between AR objects, without all of them appearing in front of the person.
Apple’s own Measure app also makes full use of the new possibilities. Measurements are carried out faster and you can now automatically measure someone’s height. There is also a new Ruler View, which allows you to take more detailed measurements. You can save all these measurements, with accompanying screenshots so that you can consult them later.
Other developers are also working with the LiDAR Scanner. The IKEA Place app has been given a new Studio Mode with which you can furnish entire rooms. The earlier app let you place a single piece of furniture in your living room, but with the new advanced Studio Mode you can decorate your entire house. This can also come in handy in games. For example, the Apple Arcade game Hot Lava has a new AR mode. In it, your own living room is transformed into a playing field full of lava.
All apps built with ARKit , Apple’s proprietary augmented reality app development tool, immediately benefit from the LiDAR Scanner. For example, with these apps it is no longer necessary to calibrate and scan the environment first. Developers of these apps do not have to do anything for this.