System performance: UL and BAPCo benchmarks

Our Q4 2022 test suite update for Windows 11 systems includes some of the standard benchmarks we’ve been using over the last few years. Although UL’s PCMark is on the list, we have decided to temporarily suspend reporting of BAPCo’s SYSmark results (until the power consumption aspect is determined). Instead, BAPCO’s cross-platform CrossMark benchmarking tool has been added to the mix along with UL’s Procyon suite. While CrossMark uses idle time compression and processes all workloads in an opaque way, UL’s Procyon processes real-world workloads through user interaction (just like SYSmark by BAPCo). We have extended the UL Procyon suite benchmark with our own custom energy measurement setup.

UL PCMark 10

UL’s PCMark 10 test evaluates computer systems for a variety of usage scenarios (general/essential tasks such as web browsing and running applications, productivity tasks such as editing spreadsheets and documents, gaming and creating digital content). We benchmarked selected PCs with the PCMark 10 Extended profile and recorded the results for various scenarios. These results are largely influenced by the processor and graphics card in the system, although RAM and storage device also play a role. The power plan was set to Balanced for all computers during PCMark 10 benchmark processing. The results for each component/use case environment are also shown below.

Intel’s latest and greatest products usually lead the charts when new products using them are introduced. Unfortunately, this does not appear to be the case with Meteor Lake UCFF computers. In the list above, the GTR7 has the 65W Ryzen 7 7840HS and easily climbs to the top position. However, even the Ryzen 7 7840U in 4X4 BOX-7840U at 40W is able to outperform Meteor Lake UCFF PCs. In fact, outside of digital content creation, none of the three Meteor Lake configurations are competing for the top spot. There are many reasons for this – including available power and clock speed budgets, which we will analyze later.

UL Procyon v2.1.544

PCMark 10 uses open source software such as Libre Office and GIMP to evaluate system performance. However, many of their professional benchmarking clients have requested evaluations of commonly used commercial software such as Microsoft Office and Adobe applications. To meet their needs, UL introduced the Procyon benchmark in late 2020. There are currently five benchmark categories – office productivity, AI inference, battery life, photo editing and video editing. AI Inference benchmarks are only available for Android devices, while battery life benchmarks apply to Windows devices such as notebooks and tablets. We present the processing results of the remaining three benchmarks.

Meteor Lake configurations perform poorly on all office workloads, losing even to NUC Wall Street Canyon. It is highly possible that the Windows Thread Director is not working as expected and tasks are being sent to high-performance or low-power cores.

It’s not much better from an energy consumption perspective either. While the absolute results place MTL systems in the bottom half of the benchmark, the energy numbers show them increasing slightly. Overall, there seems to be an increase in energy efficiency, but there’s nothing much to write home about when the Arena Canyon NUC outperforms across the board, both in terms of performance and energy efficiency.

Moving on to our evaluation of Adobe Photoshop and Adobe Lightroom, we find that the NUC BOX-155H in its 40-watt avatar has slightly greater graphics performance compared to the NUC14RVHv7. However, the results still rank in the bottom half of the pack.

While the ASRock Industrial NUC BOX-155H in its 28W configuration gets the job done with relatively low power consumption (still behind the latest AMD-based systems), the NUC14RVHv7 with 64W PL1 sits in the lower half of the pack with no performance metrics to show for it.

UL Procyon evaluates video editing performance using Adobe Premier Pro.

GPU capabilities again make the difference here, and we see the 40-watt NUC BOX-155H perform well enough to be in the top half of the pack. It still lags behind the latest AMD chips, as was seen in photo processing performance.

Meteor Lake finally wins in terms of power consumption under this workload with its 28W NUC BOX-155H configuration. The chiplet architecture seems to offer benefits here, with even the 40W and 64W PL1 configurations being in the upper half of the pack.

BAPCo CrossMark 1.0.1.86

The goal of BAPCo’s CrossMark tool is to simplify benchmark processing while still providing results roughly comparable to SYSmark. The main advantage is the cross-platform nature of the tool, which allows it to be run on smartphones and tablets.

As expected, the relative performance seen in PC Mark 10 workloads also translates to CrossMark. The situation for Meteor Lake is quite dire here – even the Alder Lake-based Wall Street Canyon NUC seems to perform better than all three MTL-H configurations tested.