AITuTu Is AnTuTu’s New Benchmark to Test Smartphones’ AI Capabilities

Share on Facebook Tweet Share Reddit Comment
AITuTu Is AnTuTu’s New Benchmark to Test Smartphones’ AI Capabilities

Highlights

  • The new AiTuTu benchmark will test AI capabilities across different SoCs
  • Manufacturers all have different approaches and levels of integration
  • AnTuTu hopes to drive the development of a common AI standard

Chinese benchmark firm AnTuTu has released a public beta version of a new benchmarking app designed to test the AI capabilities of modern smartphones. Called AITuTu, the new app is available for download as an APK file from the company's own website, but is not yet widely released. In a blog post announcing the new app, AnTuTu states that it might not actually be possible to quantify AI performance as a single score thanks to the current state of the smartphone industry, in which multiple SoC manufacturers, phonemakers and software developers are all working on their own implementations of AI. Also, some companies have not released SDKs to allow apps like this to tap into their AI capabilities, which will negatively affect the scores that some smartphones can achieve.

However, AI is growing in importance not only as a buzzword but also as a set of features that people now live with, which is why the firm decided to go ahead and release its new AI benchmark. The company says that it has established a unified standard for testing AI capabilities in cooperation with several manufacturers including Qualcomm, Samsung, MediaTek, and Nvidia and is working towards developing a way to quantify AI performance across all of their various implementations.

AnTuTu's blog post (translated from Chinese) goes on to state clearly that there will be differences between devices based on different hardware platforms. Also, manufacturers have different approaches to releasing SDKs, or tools that allow third parties to address their hardware capabilities. Samsung and Huawei in particular have not released any SDK yet, which will affect scores on their devices. The version of Android used will affect the score as well, since Android 9 has new AI optimisations.

The benchmark in its present form tests two AI scenarios – image recognition and object classification. The image classification test uses a set of 200 photos that need to be classified using a neural network called Inception V3. Object recognition parses a 600-frame video using a different neural network called Mobilenet SSD. These need to be translated into a format that each device can work with. If the benchmark is run on a device without AI acceleration the tests will fall back to using the CPU through a translation layer called TensorFlow Lite, which is not efficient and will not deliver a good score.

antutu aitutu objectrecogntition aitutu

AnTuTu also says that it has built in guards to prevent phones from cheating. The score is determined by speed and accuracy, and the latter cannot be sacrificed for the former in order to get a high speed score. The company hopes that it will help the industry reach a common standard for AI, and has stated that it will release a tool for developers to address hardware AI functions on different hardware platforms directly in order to help them target their efforts.

Gadgets 360 downloaded the AITuTu public beta and ran it on a Google Pixel 3 (Review). The APK download itself is just 33MB but the test data needs to be downloaded within the app, and it prompted us to enable Wi-Fi to do so. The image classification test showed 200 photos being classified into six categories: transportation, electonric machinery, food, animals and plants, sports scenery, and other. Although some of these categories are vague, the test seemed to do a good job of sorting the photos by content, and took about one second per photo. The app then began loading its object detection but then exited to its loading screen without any error message. This could be because the app is still a public beta release.

We also tried running the AITuTu test on a recent Samsung phone, and while the image classification test took longer to run, the object recognition test did in fact run to completion. We were shown individual scores for each test and a combined overall score. With nothing to compare these scores against yet, it was not possible to determine how much Samsung's lack of an SDK affected its showing in the benchmark.

Comments

For the latest tech news and reviews, follow Gadgets 360 on Twitter, Facebook, and subscribe to our YouTube channel.

Further reading: AnTuTu, AnTuTu AI Review, AITuTu, AI, Android
Jamshed Avari Jamshed Avari has been working in tech journalism as a writer, editor and reviewer for over eight years. He has reviewed hundreds of products ranging from smartphones and ... More
Honor View 20 With 'Hole-Punch' Selfie Camera, Up to 8GB of RAM Launched in India: Price, Specifications
Lava Z92 With Display Notch, 3,260mAh Battery Launched in India: Price, Specifications
 
 

Advertisement

 

Advertisement