AnandTech: MediaTek-based smartphones have cheat mode for benchmarks

Posted on Thursday, April 09 2020 @ 14:50 CEST by Thomas De Maesschalck
AnandTech investigated why certain MediaTek-based smartphones score so high in performance tests. What tipped the site of was that phones like the Oppo’s new Reno3 Pro, which use the MediaTek Helio P95 chipset, seem to punch far above their weight. AnandTech was able to get an anonymised version of PCMark from UL Benchmarks and what they discovered is that with the anonymised benchmark, performance was about 25 percent lower. You can check out the article over here.

As it turns out, the Reno3 Pro has a "Sports Mode" benchmark whitelist hidden in plain view. This Sports Mode artificially increases performance for a number of benchmarks like AnTuTu, GeekBench, PCMark, 3DBench, and others. AnandTech checked more MediaTek-based phones from Oppo, Vivo Xiaomi, Realme, iVoomi and Sony. They discovered that each device they tested had a "power_whitelist_cfg.xml" file in the vendor partition.

AnandTech notes this cheating mechanism isn't stemming from the individual phone vendors, but directly from MediaTek. The site reached out to MediaTek and received the following statement. The company doesn't deny the cheating mechanism but calls it a way to reveal "the upper end of performance capabilities."
MediaTek Statement for AnandTech
MediaTek follows accepted industry standards and is confident that benchmarking tests accurately represent the capabilities of our chipsets. We work closely with global device makers when it comes to testing and benchmarking devices powered by our chipsets, but ultimately brands have the flexibility to configure their own devices as they see fit. Many companies design devices to run on the highest possible performance levels when benchmarking tests are running in order to show the full capabilities of the chipset. This reveals what the upper end of performance capabilities are on any given chipset.

Of course, in real world scenarios there are a multitude of factors that will determine how chipsets perform. MediaTek’s chipsets are designed to optimize power and performance to provide the best user experience possible while maximizing battery life. If someone is running a compute-intensive program like a demanding game, the chipset will intelligently adapt to computing patterns to deliver sustained performance. This means that a user will see different levels of performance from different apps as the chipset dynamically manages the CPU, GPU and memory resources according to the power and performance that is required for a great user experience. Additionally, some brands have different types of modes turned on in different regions so device performance can vary based on regional market requirements.

We believe that showcasing the full capabilities of a chipset in benchmarking tests is in line with the practices of other companies and gives consumers an accurate picture of device performance.


About the Author

Thomas De Maesschalck

Thomas has been messing with computer since early childhood and firmly believes the Internet is the best thing since sliced bread. Enjoys playing with new tech, is fascinated by science, and passionate about financial markets. When not behind a computer, he can be found with running shoes on or lifting heavy weights in the weight room.



Loading Comments