UL kicks Huawei phones out of it database due to cheating

Posted on Friday, September 07 2018 @ 12:12 CEST by Thomas De Maesschalck
UL logo
3DMark owner UL Benchmarks has delisted several Huawei phones from its database due to cheating.

Basically, Huawei created a software path to detect when a user ran a benchmark like 3DMark on its devices. While typical workloads on the phones are subject to thermal throttling, this is not the case for a series of whitelisted benchmarks. This results in artificially high, and misleading, benchmark scores.

This behavior is against the rules of UL, and resulted in the delisting of the Huawei P20 Pro, Nova 3, Play, and P20 smartphones. UL performed independent testing with the publicly available 3DMark app and a renamed version that could not be identified by the Huawei phones. In case of the public app, performance was up to 47 percent higher due to the use of a "Performance Mode" power profile that overrides the smartphones' usual power profile.

In a reply to UL, Huawei commented that they intend to make this "Performance Mode" publicly available:
The difference in scores tells us that the devices are simply recognizing the 3DMark app by name rather than adapting to the type of work in the test.

This kind of detection and optimization is forbidden by our rules for manufacturers. We contacted Huawei with our findings, and we are happy to say that they have pledged to implement a more transparent approach in a future update,

"Huawei is planning to provide users with access to “Performance Mode” so they can use the maximum power of their device when they need to."

Optional performance modes that can be set by the user—already available on some other manufacturers' models—are allowed under our current rules as long as they are disabled by default. A device must run the benchmark as if it were any other application.
Huawei cheating

Interestingly, this is the sort of behavior we saw from PC CPU and GPU makers many many years ago.


About the Author

Thomas De Maesschalck

Thomas has been messing with computer since early childhood and firmly believes the Internet is the best thing since sliced bread. Enjoys playing with new tech, is fascinated by science, and passionate about financial markets. When not behind a computer, he can be found with running shoes on or lifting heavy weights in the weight room.



Loading Comments