In the blink of an eye, apple’s ARKit is already in ARKit 4, and in a few months ‘time, WWDC2021 will see ARKit 5. I have no doubt about the future of THE AR industry, but so many years have passed, the main force of AR is still mobile AR, and the VR industry next door has popularized VR headset compared to the current market AR glasses are like toys in toys.

I also keep thinking, is the current mobile AR the right path? What exactly is it good for?

Mobile AR status

Some people in the industry have said: THE basic technology of AR has been available, but the main problem is that mobile phones are not the ideal carrier of AR. The current embarrassment of mobile AR is basically the same.

The main disadvantages are:

  • Input and output: display through the mobile phone screen is not intuitive, input through the screen is not convenient;
  • Performance and battery life: The current performance and battery life of mobile phones are far from enough if AR is continuously used;
  • Use form: need to use the hand to hold, use inconvenience;


After thinking about it, I think the application of mobile AR industry at present mainly has the following directions:

Low version of VR

In other words, the mobile phone AR is treated as a low-configuration VERSION of VR, and the mobile phone is used for 6 DoF tracking, 3D preview, gesture click, drag and drop, daub and other basic interactive operations. Although the current VR hardware has high maturity and good stability, its popularity is not high due to price and portability.

Common features of such apps are:

  • Point the phone at a flat ground or a specified card, place a virtual object on it, and let the virtual object play an animation, or tap the screen for simple interaction;
  • Due to the limited stability of mobile phone AR, it is unable to carry out fierce confrontation and precise operation.
  • With the power of Unity/UE, it’s easy to get good visuals quickly, but performance is often poor (this isn’t Unity’s fault, it’s because you don’t have the money or the time to polish them carefully);
  • Low correlation with reality. Although AR is augmented reality, such apps mainly use the 6 DoF tracking function of AR to display 3D content more conveniently.

AR filter

AR filter App is different from the above, and has a deep connection with reality. Ordinary filters generally process flat pictures and videos, while AR filters can create a view with depth and try to understand the object seen in the lens: is it a human body? Is it a face? How far is it from the phone? Is it a plane?

The common features of such apps are:

  • Convenient integration with existing technologies and businesses, such as AR filters in video software, AR makeup and fitting in shopping software;
  • Related to the human body or the surrounding environment, face makeup try, clothes try, furniture try, etc.
  • Theories can create cool visuals, but it’s still technically difficult to fully understand the real world, so scenes and interactions tend to be fixed;

Scan the modeling

With the introduction of ToF lenses on mobile phones, it is now common to use mobile AR capabilities for general scan modeling. The common features of such apps are:

  • It has certain practical value, for example, scan the 3D model of the recorded room to automatically recommend furniture, estimate the size and volume of the object, and automatically select the appropriate size of the carton;
  • Still limited by the accuracy and computing ability, to complete a good effect is slow, real-time processing effect is poor;
  • Requires a certain amount of algorithmic or AI capability to achieve good results;

Guidance and positioning

This function should be the concept of AR fire up, the first to be thought of, but it is very difficult to achieve, so now is not good, such as AR navigation.

The current characteristics of such apps are:

  • Positioning is still problematic, whether it is GPS, Bluetooth, UWB, VIO, video frame matching, cloud anchor points, etc., either large range and high precision can not be both, or indoor hardware needs to be deployed in advance;
  • It requires AI and cloud support, which can’t be done well with a phone camera alone;
  • Energy consumption is amazing, GPS+ Bluetooth + camera SLAM+ network communication +3D rendering + screen steady on, all are big power consumption;
  • It is very suitable for integration with existing map products. Baidu Map and Autonavi Map have launched AR navigation function.

The last

At present, various technical routes and basic functions required by AR products have been implemented on mobile phones. However, mobile AR itself is full of shortcomings, and is not an ideal AR carrier.

At present, various manufacturers are mainly developing and probing:

  • Technology verification: The current mobile AR, whether mobile phone manufacturers or App developers, seems to regard mobile AR as a verification platform. Verify that hardware, algorithms, and business aspects can work;
  • Train the team: verify the technology and ideas, but also play the role of training the team;
  • Educate users: attract users with mobile AR, popularize AR concepts and some AR gameplay to users, and feedback user ideas;