Performance By Design

Recently, I have been writing about how Windows guest machine performance counters are affected by running in a virtual environment, including publishing two recent, longish papers on the subject, one about processor utilization metrics and another about memory management. In the processor utilization paper, (which is available here ), it is evident that running under VMware, the Windows performance counters that measure processor utilization are significantly distorted. At a system level, this distortion is not problematic so long as one has recourse to the VMware measurements of actual physical CPU usage by each guest machine. A key question – one that I failed to address properly, heretofore – is whether it is possible to correct for that distortion in the measurements of processor utilization taken at the process level inside the guest machine OS. The short answer for Windows, at least, is, “Yes.” The % Processor Time performance counters in Windows that are available at the proces...

This is a continuation of a series of blog posts on VMware memory management. The previous post in the series is  here . Ballooning Ballooning is a complicated topic, so bear with me if this post is much longer than the previous ones in this series. As described earlier , VMware installs a balloon driver inside the guest OS and signals the driver to begin to “inflate” when it begins to encounter contention for machine memory, defined as the amount of free machine memory available for new guest machine allocation requests dropping below 6%. In the benchmark example I am discussing here, the Memory Usage counter rose to 98% allocation levels and remained there for duration of the test while all four virtual guest machines were active. Figure 7, which shows the guest machine Memory Granted counter for each guest, with an overlay showing the value of the Memory State counter reported at the end of each one-minute measurement interval, should help to clarify the state of...

Performance testing is one of the most important practices associated with applying software performance engineering principles to acceptance testing and other quality assurance processes. This blog entry discusses the key role scalability models play in performance testing. A scalability model hypothesizes a relationship between a performance-oriented response time or throughput goal for a scenario and one or more scalability dimensions associated with that specific scenario. One such scalability dimension, the arrival rate of customer requests, is a staple of performance stress tests, for example. One of the main assertions here is the value of making explicit the scalability assumptions that are implicit in defining performance acceptance testing criteria. Assumptions about the dimensions of an application’s scalability inform the definition of an appropriate performance test matrix to ensure the testing process is both rigorous and effective. Furthermore, feedback from empi...

This blog post describes the Scenario instrumentation library, a simple but useful tool for generating response time measurements from inside a Windows application. The Scenario instrumentation library uses QPC() and QTCT(), the Windows APIs discussed in an earlier blog entry , to gather elapsed times and CPU times between two explicit application-designated code markers. The application response time measurements are then written as ETW events that you can readily gather and analyze.  You can download a copy of the Scenario instrumentation library here at http://archive.msdn.microsoft.com/Scenario . The Scenario class library was originally conceived as a .NET Framework-flavored version of the Application Response Measurement (ARM) standard , which was accepted and sponsored by the Open Group. The idea behind ARM was that adding application response time measurements to an application in a standardized way would promote 3rd party tool development. Thi...