Futuremark Delists Android Devices Allegedly Cheating 3DMark, Samsung and HTC Ousted
Benchmarking is serious business, and we're not saying that tongue-in-cheek. Buying decisions are often made based on how well a product benchmarks, which is why we spend so much time putting hot (and sometimes not-so-hot) hardware through their paces. However, benchmarks are only meaningful when there's a level playing field, and when companies try to 'game' the business of benchmarking, it's not only a form of cheating, it also bamboozles potential buyers who (rightfully) assume the numbers are supposed mean something.
We bring this up because Futuremark just delisted a bunch of devices from its 3DMark benchmark because it suspects foul play is at hand. Here's a look at the devices in question:
"People rely on Futuremark benchmarks to produce accurate and unbiased results. That's why we have clear rules for hardware manufacturers and software developers that specify how a platform can interact with our benchmark software," Futuremark said in statement. "In simple terms, a device must run our benchmarks without modification as if they were any other application."
Futuremark updated 3DMark for Android, which you can grab from Google Play. In addition to delisting suspicious devices, the new versions now allows you to filter scores by color-coded OS in the Device Channel and improves compatibility with the lastest NVIDIA hardware and some Samsung Galaxy S3 models.
We bring this up because Futuremark just delisted a bunch of devices from its 3DMark benchmark because it suspects foul play is at hand. Here's a look at the devices in question:
Delisted devices are stripped of their rank and scores. In this instance, several Samsung and HTC devices are accused of breaking Futuremark's benchmarking rules. Futuremark didn't elaborate on which specific rule(s) these devices broke, but a look at the company's benchmarking policies reveals that hardware makers aren't allowed to make optimizations specific to 3DMark, nor are platforms allowed to detect the launch of the benchmark executable (unless it's needed to enable multi-GPU and/or there's a known conflict that would prevent it from running).
Futuremark updated 3DMark for Android, which you can grab from Google Play. In addition to delisting suspicious devices, the new versions now allows you to filter scores by color-coded OS in the Device Channel and improves compatibility with the lastest NVIDIA hardware and some Samsung Galaxy S3 models.
