There has been plenty of fanfare about the arrival of the latest incarnation of Windows*, called Windows* Vista*. Highlights include a new GUI, new APIs, and new search capabilities. But new features can bring with them new challenges, especially for performance measurement.

Experienced benchmarkers may recall some of the initial issues during the early days of Windows* XP. There were issues involving that OS's features to optimize user experience, and background services possibly introducing variability into test results. Eventually, these issues got ironed out, but with the arrival of Windows* Vista*, we thought we'd share our best known method (a "BKM" in Intel-speak) for benchmarking on Vista*.

So without further ado, here's how we benchmark Windows* Vista* in our labs.

Getting Started

We recommend this setup procedure for all benchmarking under Microsoft* Windows* Vista* Ultimate Edition.

• Always start with a clean, formatted hard disk.

• If your motherboard supports raid configurations with Intel® Matrix Storage Technology, then set the hard drive configuration in the bios to RAID to make your system raid-ready. Using this setting in conjunction with the Intel® Matrix Storage Driver can provide better system performance because features like Native Command Queuing and I/O request sorting will be enabled even with a single hard drive installed in the system.

• Set your system to boot from CD/DVD drive in the bios.

• Begin installation Windows* Vista* Ultimate Edition* from the operating system CD.

• Format the file system to NTFS during installation if the drive if you have not already done so.

• After installation of the OS is complete restart your system.

• Immediately after installing the Windows* Vista* operating system, install the latest INF files to allow the operating system to recognize the chipset and all the components on the motherboard. The INF file can be downloaded from the chipset software link at http://downloadfinder.intel.com/scripts-df-external/Support_Intel.aspx?lang=eng

• Install the latest Intel® Matrix Storage Manager. The install file can be downloaded from the chipset software link at http://downloadfinder.intel.com/scripts-df-external/Support_Intel.aspx?lang=eng . (note: Windows* Vista* installs the basic driver for Intel Matrix Storage, and you should update the driver once the OS boots up.) The Intel® Matrix Storage Driver can provide better system performance because features like Native Command. Some system benchmarks with an I/O component, like BAPCo* SYSmark*, may show increased performance with the Intel Matrix Storage Driver installed and the system set to raid-ready.

• Install the Intel integrated sound driver from http://support.intel.com or from your motherboard CD.

• Install the latest LAN driver from http://support.intel.com.

• Download and Install the latest Windows* Vista* graphics driver for your graphics card from the manufacturer website.

Windows* Prefetch

Windows* Prefetcher places files for your most frequently used applications in a cached folder so they will load faster. This feature was introduced in Windows* XP, improved in Windows* Vista*. While a useful consumer feature, it can affect benchmark results.

We recommend cleaning the Windows* Prefetch folder by deleting the contents of this folder:

c:\windows\prefetch

This only needs to be done between testing of individual benchmarks. We generally recommend running benchmarks five times in Windows* Vista* and taking the median score. You may notice your scores improving slightly in latter runs, as the Windows* Prefetcher* gets application files into the Prefetch folder.

System Settling

To allow your system to settle into an idle state:

1. Reboot your computer.

2. Open the Task Manager by right clicking on the Taskbar and clicking the option for Task Manager. Click on the Performance tab in the Task Manager.

3. Click on the Resource Monitor button in the Task Manager. This will bring up the Resource Monitor Window below.

4. Click on the Pane that says Disk. This will allow you to view the various active requests to the disk drive.

5. Watch the Resource Monitor. The number of requests to the disk will gradually decline. Wait until there are no more requests to the disk as shown below.

6. After there are no more requests to disk, close the Resource Monitor.

Next, execute Process Idle Tasks by calling the ProcessIdleTasks API from advapi32.dll from the command line, or create and execute a batch file with the following command:

rundll32.exe advapi32.dll,ProcessIdleTasks

The table on the next page describes the how to perform an initial setup of Windows* Vista.

Recommended Settings

Microsoft* Windows* Vista* Setup	Setting	Description of how to set up Windows* Vista
Windows* Aero	Enable	Ensure Windows* Aero* is enabled - Right click on desktop -> Personalize -> Window color and appearance -> click on Open Classic Appearance Properties for more color options - Ensure Windows* Aero* is checked. Note: In order for Aero* to work on Intel Graphics Media Accelerator X3000 and 3000 Vista* drivers should be installed prior to setup Aero* feature in Vista.
Search Indexing	Enable	Leave Search Indexing Enable - You can check if Search Indexing is enable by Open Control Panel -> System and Maintenance -> Indexing Options. The Indexing location will include: Offline Files Start Menu and Users.
Internet Offline Files	Enable	Leave Internet Offline files Enable - You can check if Offline files are enabled Open Control Panel -> Network and Internet -> Offline Files. On the general tab the Disable Offline Files button should appear.
Disable Screen saver	Disable	Right click on desktop -> Personalize -> Screen Saver. Set the screen saver to None
Disable Power management	High Performance	Right click on desktop -> Personalize -> Screen Saver -> Change Power settings -> Select High performance and click Change plan settings below this option. A new view will appear called Edit Plan Settings. Change Turn off display to Never. Click Save Change.
Vista* Desktop Display resolution and monitor refresh	Custom	Right click on desktop -> Personalize -> Display Settings. In the Display Settings dialog choose Resolution=1280x1024 and Colors= Highest (32bit). Click the Advanced Settings button. Select the Monitor tab and change the Screen refresh rate to 75 Hz.
Remove always on top	Disable	Right click on Windows* task bar at the bottom -> Properties -> Taskbar -> uncheck Keep the taskbar on top of other windows
Remove hide inactive icons	Disable	Right click on Windows* task bar at the bottom -> Properties -> Select the Notification area tab and uncheck Hide inactive icons
Disable Windows* Defender	Disable	Open Control Panel -> Security -> Windows* Defender. Click on Tools in the toolbar at the top. Click on the Options link in the next view. Uncheck all boxes.
Disable Windows* Welcome screen	Disable	Open Control Panel -> System and Maintenance -> Welcome Center. Uncheck Run at Startup at the bottom of the Windows* Welcome screen
Disable System Protection (previously System Restore)	Disable	Open Control Panel -> System and Maintenance -> System. Under the tasks link on the left side select System Protection. Uncheck Local Disk found in the Automatic restore points section.
Disable Windows* Update	Disable	Open Control Panel -> Security -> Windows* Update. Click on the change settings link on the left side. Select Never check for updates (not recommended)

Running Benchmarks

Your system should now be ready for benchmarking. We recommend running workloads five (5) times in a row after you Process Idle Tasks without reboot in between, and then taking the median score of the five runs.

Gentlemen, Start Your Benchmarks

Now, it's your turn. We'd like to hear from you about your adventures (and misadventures) benchmarking with Windows* Vista. Today, industry benchmarks like PCMark05* and 3DMark06* can be patched so that they run on Windows* Vista*. Most Windows* XP applications will run on Vista*, so application-based tests are another good testing option.

Stay tuned for some upcoming Vista*-ready Software Evaluation Guides (SEGs) coming here soon.

We're also putting the finishing touches on Intel® DHCAT 2.0, which runs on both Windows* XP and Windows* Vista*.

Head over to our forums and let us know what you think of Windows* Vista*, why you want (or don't want) to upgrade to it, and how you plan to test on it.

* Other names and brands may be claimed as the property of others