Geekbench ML 0.5, the first preview release of Primate Labs’ new machine learning benchmark, is now available for Android and iOS. Geekbench ML is a free download from Google Play and the App Store.
Machine Learning Benchmark
Geekbench ML measures machine learning inference (as opposed to training) performance using real-world computer vision and natural language processing tasks. We based these tasks on real-world machine learning tasks found in cutting-edge mobile applications. For example, here are some of the applications and tasks we considered when designing Geekbench ML:
- Video conference applications use Image Segmentation to provide features like automatic background blur.
- Social media applications use Face Detection (along with facial landmark recognition) to provide features like virtual makeup filters and facial modification filters.
- Exercise applications use Pose Estimation to detect body posture and provide feedback and guidance on specific exercises.
- Applications use Machine Translation to provide real-time on-device translation of web pages, text messages, and even text in pictures.
All of the tests use well-understood industry-standard models, ensuring Geekbench ML results are relevant and applicable to mobile machine learning application performance.
Complete descriptions of the workloads, including information about the datasets, the model structure, and the accuracy metrics, are available in the Geekbench ML 0.5 Inference Workloads document.
Cross Platform Comparisons
We designed Geekbench ML from the ground up for cross-platform comparisons. Each test uses the same models and the same test dataset across every platform, allowing users to compare machine learning performance across devices, operating systems, and accelerators.
Geekbench Browser Integration
Geekbench ML also integrates with the Geekbench Browser, allowing users to upload and share their Geekbench ML benchmark results with other users worldwide.
We are also publishing a Machine Learning Benchmark Chart based on Geekbench ML results uploaded to the Geekbench Browser. This chart enables users to see how their devices should perform and how devices stack up against one another.
Preview
Geekbench ML 0.5 is a preview release. We hope to iterate rapidly on Geekbench ML and its benchmarks this year, with a final Geekbench ML 1.0 release later this year. This rapid iteration will allow us to incorporate community feedback and improve the benchmark faster. It will also mean that preview release scores may not be compatible with one another. For example, we do not expect Geekbench ML 0.5 scores will be compatible with Geekbench ML 0.6 scores.
We’ve been working on Geekbench ML for over two years. We’re incredibly excited about this release, and we hope you are, too. If you have any questions or comments about Geekbench ML, please let us know on the Geekbench Support Forum.