AquaMark3 - GeForce FX 5900 Ultra Vs. Radeon 9800 Pro
DX8/DX9 Pixel and Vertex Shader Benchmarking

By: Dave Atlavilla
September 15, 2003



 

All of the recent controversy over Valve's Half Life 2 presentations at ATi's Shader Day, relative to NVIDIA's GeForce FX performance (or lack thereof), has prompted many of us here at HotHardware to try and further substantiate GeForce FX and Radeon 9XXX performance in DirectX 9 gaming scenarios.  Meanwhile, the folks at Massive Development have been hyping the upcoming release of their 3rd generation game engine as a benchmark, AquaMark3.  AQM3 is based on Massive's "Krass" engine which certainly can render impressive game scenery.  So, it was with great interest that we recently received the AquaMark3 beta release, for evaluation and comment on its launch day.  Could this benchmark, with its claimed use of DX9 Pixel and Vertex Shader effects and its tag line, "the reality benchmark", assist us in the appraisal of current generation NVIDIA and ATi DX9 hardware?  We were eager to find out.

Although, the coming out of Valve's Half Life 2 struggle with the GeForce FX line of products, certainly was a splash in a major league sort of way, one game engine is hardly a basis for trashing an entire product line.  So, with that in mind, we set out to look at DX9 performance from yet another data-point, with Massive's newly claimed "reality benchmark".  Does AquaMark3 have the ability to measure a true DX9 gaming experience?  Let's peel back the onion a bit (no Futuremark puns intended) and find out!

Menu Options, Controls and Features
A good tool for benchmarking, easy, configurable and full of features
 
AquaMark3 is a very well equipped benchmark.  There are a number of ways to run a test or a series of tests, with either custom settings or standard configurations.  The freeware version of the software, allows users to run only the default 1024X768 benchmark or "TRISCORE", which reports back a total AquaMark score, as well as system and CPU scores.  Users then have the ability to upload their scores to an online database engine called "ARC", the AquaMark3 Result Comparator.
 

Measurements

Special Features

 
Advanced Options

 
ARC View

The professional and commercial versions of the benchmark, allow you the ability to dial in specific settings for resolution, color depth, AA, Aniso Filtering etc.  In addition, within these versions, you also have the ability to run the "Special Features" tests and utilities.  A screen capture utility is available, as well as Overdraw Visualization,  Shader Visualization and Pixel Performance Measurement scripts.  We took some time to look at the Shader Visualization feature, since it provides a very easy way to see the use of the engine's various Pixel Shader implementations, through the use of colorized overlays, which highlight the types of shaders used in a given scene.  Blue colored pixels represent no pixel shader effects, yellow pixels represent PS 1.x effects and red pixels indicate PS 2.0 (DX9) effects.
 
Speaking of which, let's take a look at they various types of Pixel Shader effects that are utilized in AquaMark3.

 
Shader Specifications Of AquaMark3
A DX9 benchmark or a mixed bag of DX8 and DX9?

The table above shows exactly the number and types of Pixel and Vertex Shaders utilized in AquaMark3.  Look at the last column on the right.  Herein lies the rub, so to speak.  Some quick addition tells you that there are 7 DX9 Pixel Shader or Vertex Shader 2.0 effects, in this benchmark.  There are also 225 other PS or VS 1.x shaders, used in conjunction with the 7 PS and VS 2.0 shaders.  This seems pretty light for a benchmark that claims to be a "DX9 benchmark".  However, Massive did inform us that about 30% of the pixels rendered in the AquaMark3 benchmark, are rendered with DX9 shader effects.  Regardless, we feel that based on this realization, that AquaMark3 is much more of a blended DX8/DX9 benchmark, rather than a "pure" DX9 benchmark per se.  Massive calls AquaMark3 a "reality benchmark" and perhaps for now it is. 

Current game technology in the market today is based on a mix of different shader types.  However, just as DX7 effects fell to the wayside to make room for DX8, so too will DX8 shaders in favor for DX9.  It's just a matter of time.  Half Life 2 for example uses about 1200 or so DX9 PS 2.0 Shaders and Tomb Raider Angle of Darkness is cut from the same cloth.  So perhaps Massive is a bit over zealous in their claim that AQM3 is a DX9 benchmark.  For certain, AQM3 utilizes DX9 shaders and delivers a well rounded benchmark suite, with scoring based on those performance metrics.  However, we feel it falls short of being a true DX9 benchmark.  Even 3DMark03's "Mother Nature" test, which has nearly 100% coverage of DX9 shaders to render the pixels in its scenery, may be a better indicator of DX9 performance it seems.

Screen Shots And Image Quality