How the Methodology Works

This methodology specifies a workday modeled around four SYSmark* 2004 SE runs. For cost of benchmarking purposes, one run of SYSmark can be measured and used as a representative sample for the rest of the methodology.

That said, general best practice with any benchmark is to run it three times and take the median value of the three runs. That practice applies here.

In addition to obtaining a performance score, the system’s power draw level from the wall (AC socket) will need to be recorded as well. There are several ways to obtain this data. This can be done using a simple, inexpensive watt-meter, or using a more sophisticated meter that supports data-logging. The latter piece of equipment will make the task of gathering system power data easier, and will yield the more accurate results.

The next consideration is the watt-meter’s sampling rate. Low-cost AC watt-meters do not include the ability to log power draw over time. More costly tools may support logging, but if an insufficient sampling rate is used, many transients (spikes and troughs) may not be captured and could skew average power measurements. Intel’s measurements presented here used a logging sample-rate of 2Hz (2 samples per second).

More expensive watt-meters enable automatic logging of data, where a tester would only have to start the meter before beginning the SYSmark* run, and then stop it after the Internet Content Creation run completes at which time the PC will automatically reboot. After rebooting, a tester would restart the meter at the beginning of the Office Productivity run and stop it after the run completes. This reboot time should not be counted as part of the power measurement. This also helps ensure that BAPCo* rules are followed. It is not part of the performance measurement period and not part of what BAPCo* intends to model with the workload. The information that will need to be captured includes:

• The time it takes SYSmark Internet Content Creation to run (may be captured by stop watch or the watt-meter).
• The average power consumed during this time (captured by the watt-meter automatically or something that will have to be calculated manually from the captured sample).
• The time it takes SYSmark Office Productivity to run (may be captured by stop watch or the watt-meter).

The average power consumed during this time (captured by the watt-meter automatically or something that will have to be calculated manually from the captured sample).

To ensure stability of measurements, testers may want to perform more than one measurement. One recommendation is to take three runs and use the data from the run that the median performance.

Next, measure the idle and Standby system power draw levels.

For idle, just let the system sit with no active applications running on it, and take samples over a three minute period, and then calculate average idle power from those samples.

To measure Standby system power draw, go to the Windows* Start Menu, and select Shut Down. Then select Standby and the system will go to Standby mode. Take samples over a three minute period, and then calculate average idle power from those samples.

This methodology can be applied to platform evaluation irrespective of the platform’s energy savings settings. Note however that for best energy efficiency, users should enable features like Enhanced Intel® SpeedStep™ Technology (EIST), whose benefits include achieving optimal idle power energy-efficiency. EIST exists to help with energy efficiency, and if this is a primary concern, then this technology should be enabled. With that in mind, tested platforms presented here had EIST enabled.

Mobile Platform Considerations

If the system being evaluated is a mobile platform, an industry-based benchmark is already available called MobileMark* 2005. This methodology is not meant to replace MobileMark* 2005, as that benchmark already conveys the relevant energy efficiency of a mobile platform, which is its battery life.

However, this methodology could be used to evaluate a mobile platform’s on-AC (plugged into the wall) energy-efficient performance. The system should be connected to an external monitor, and the system’s graphic sub-system configured to display only on the external monitor. The backlight in mobile platforms is the biggest consumer of power, and by removing it from the evaluation “equation,” a tester can get a clearer picture of the power draw characteristics of the rest of the platform (CPU, chipset, RAM, disks, etc.). This approach will provide a better comparison between desktop and mobile platforms because monitor power is not included in the desktop calculations and removing the LCD monitor avoids burdening the notebook with energy used by a display device where the desktop measurement excludes the energy used by a display device. Alternatively, a tester could first evaluate the platform configured as described here, and then retest the system using the integrated display to understand the difference between the two configurations.

Finally, the laptop’s battery should be left in the system and should be fully charged prior to beginning the SYSmark* run, as some system’s AC power adapters cannot handle the transient power demand spikes of the system, and the battery effectively augments the available power level to cover these spikes. Such a system will not run unless the battery is present.