The guys from Tom's Hardware offer a lengthy article in which they test whether the Windows Vista Performance Indices can replace regular benchmarks:
There is a lot of variety in the hardware market: hundreds of types of processors, motherboards, hard drives, graphics cards and other components to choose from. And although consolidation has been going on for several years, leaving only AMD/ATI, Intel and Nvidia as the main players, along with a few motherboard makers and storage giants, finding the right products still can be an exhausting task. This is where benchmarking comes into play, because it helps to measure qualities and characteristics by applying a certain metric. Windows Vista now provides a built-in benchmarking solution to assess component and system performance, but does the so-called Experience Index correspond to what Tom's Hardware and other tech publications find on the test bench?
Check it out over here. After lots of testing they conclude that the Windows Experience Index is welcome, but that it definitely can't replace traditional benchmarking.