Getting more out your Windows Performance Counters monitoring for web applications – part 3

You are here: Sysadmins of the North » Windows Server » Getting more out your Windows Performance Counters monitoring for web applications – part 3

This is part 3 about Zabbix monitoring for your websites and ASP.NET applications in IIS. This time I’ll show you how to get data from Win32_PerfRawData_PerfProc_Process counter, fast, for everey application pool This counter is notorious for its slowness, but you can get data a bit faster. IIS AppPool Insights in Zabbix – because there is always more than one way.

In part 1 I showed you how to create an IIS website autodiscovery item and application discovery, and in part 2 you monitor the performance of your ASP.NET web application utilizing Windows performance counters in Zabbix. Now we take it a step further, to monitor Win32_PerfRawData_perfProc_Process data for all currently running application pools (autodiscovered).

I query PerfRawData, because using Win32_PerfFormattedData_perfProc_Process costs a small performance hit, and results may be rounded to 0 and not 0,1.

Autodiscover running IIS application pools

In this part you’ll query Win32_PerfRawData_PerfProc_Process performance counters using WMI/CIM in PowerShell. The Win32_PerfRawData_PerfProc_Process class contains a lot of usefull properties for your IIS website application pools.

The gathered data needs to be transformed into a JSON blob so Zabbix can use it, but you also need to somehow match your application pool’s (w3wp.exe) process ID to the result of your WMI query (called “WQL”, or SQL for WMI. See Querying with WQL, WQL (SQL for WMI) for more information). This is done in PowerShell.

Let’s begin.

Create a file called w3wp-discovery.ps1 and add the following into it:

$allpools = (Get-CimInstance -EA SilentlyContinue -ClassName applicationpool -Namespace root\webadministration).Name

This outputs simple all applications pools. Now you’ll need to add a Zabbix MACRO {#APPPOOLNAME} object.

@{ "data" = $allpools | foreach { @{ "{#APPPOOLNAME}" = $_ }} } | ConvertTo-Json
Code language: PHP (php)

This’ll give you an object $mypools with all discovered IIS application pools and a macro {#APPPOOLNAME}. It looks like:

{ "data": [ { "{#APPPOOLNAME}": "DefaultAppPool" }, { "{#APPPOOLNAME}": ".NET v4.5 Classic" }, { "{#APPPOOLNAME}": ".NET v4.5" }, { "{#APPPOOLNAME}": "example.com" }, { "{#APPPOOLNAME}": "example.org" }, ....
Code language: JSON / JSON with Comments (json)

The macro {#APPPOOLNAME} is used to reference application pools in your autodiscover template. We need this information later on.

Zabbix conf

For Zabbix, create a configuration file w3wp.conf in your zabbix_agentd.confd folder, and add the following UserParameter to it:

UserParameter=w3wp.discovery,powershell -NoProfile -ExecutionPolicy Bypass -File C:\path\to\w3wp-discovery.ps1

This creates the w3wp.discovery key.

Counter query Powershell script for dependent items

To query Win32_PerfRawData_perfProc_Process for performance counter values, we need PowerShell. You can use Get-WmiObject of Get-CimInstance cmdlets, and the data I want to monitor for application pools is (you are free to choose your own of course).

  • HandleCount
  • IDProcess
  • IOReadBytesPerSec
  • IOReadOperationsPerSec
  • IOWriteBytesPerSec
  • IOWriteOperationsPerSec
  • PercentProcessorTime
  • PrivateBytes
  • ThreadCount
  • WorkingSet
  • WorkingSetPeak

This is where the magic happens and you get all that info:

$processinfo = Get-CimInstance -EA SilentlyContinue -Query "select * from Win32_PerfRawData_PerfProc_Process where Name like 'w3wp_%'" -Namespace root\cimv2 | Select HandleCount,IDProcess,IOReadBytesPerSec,IOReadOperationsPerSec,IOWriteBytesPerSec,IOWriteOperationsPerSec,PercentProcessorTime,PrivateBytes,ThreadCount,WorkingSet,WorkingSetPeak
Code language: PHP (php)

The WQL queries Win32_PerfRawData_PerfProc_Process for all data, and then I use PowerShell Select to filter the output. Why? Because this is way faster than running a query per application pool like "select * from Win32_PerfRawData_PerfProc_Process where IDProcess='w3wp.exe PID'". Especially if you need to this 300 or 500 times on a server. WMI is really slow, so let PowerShell do the filtering of data.

The output may look as follows:

HandleCount : 2537 IDProcess : 7652 IOReadBytesPerSec : 123547753 IOReadOperationsPerSec : 112674 IOWriteBytesPerSec : 78788008 IOWriteOperationsPerSec : 88060 PercentProcessorTime : 4461250000 PrivateBytes : 278888448 ThreadCount : 159 WorkingSet : 283172864 WorkingSetPeak : 288657408 HandleCount : 3038 IDProcess : 5156 IOReadBytesPerSec : 27230185458 IOReadOperationsPerSec : 3777431 IOWriteBytesPerSec : 87141280 IOWriteOperationsPerSec : 89040 PercentProcessorTime : 248751718750 PrivateBytes : 1265516544 ThreadCount : 183 WorkingSet : 1267384320 WorkingSetPeak : 1358626816

It doesn’t look like something you can work with in a Zabbix template, but fortunately you can use PowerShell to transform this in a nicely formatted JSON. This is why we need the IDProcess property; it get’s matched to an application pool’s .ProcessId property.

The following funcion get’s called in a ForEach loop to select the correct data for each ProcessId / IDProcess.

function get-WmiProcess($wpname) { Try { $procid = ($allprocids | Where-Object AppPoolName -eq "${wpname}").ProcessId $data = ($processinfo | Where-Object IDProcess -eq $procid) return @{ IOReadOperationsPerSec = $data.IOReadOperationsPerSec IOWriteOperationsPerSec = $data.IOWriteOperationsPerSec IOReadBytesPerSec = $data.IOReadBytesPerSec WorkingSetPeak = $data.WorkingSetPeak HandleCount = $data.HandleCount ThreadCount = $data.ThreadCount IOWriteBytesPerSec = $data.IOWriteBytesPerSec PercentProcessorTime = $data.PercentProcessorTime WorkingSet = $data.WorkingSet WorkingSetPrivate = $data.WorkingSetPrivate PrivateBytes = $data.PrivateBytes } } Catch { } }
Code language: PHP (php)

And the ForEach loop to get the data and eventually output the JSON:

$processinfo = (Get-CimInstance -EA SilentlyContinue -ClassName Win32_PerfRawData_PerfProc_Process -Filter "Name like 'w3wp%'") | select HandleCount,IDProcess,IOReadBytesPerSec,IOReadOperationsPerSec,IOWriteBytesPerSec,IOWriteOperationsPerSec,PercentProcessorTime,PrivateBytes,ThreadCount,WorkingSet,WorkingSetPeak,WorkingSetPrivate $procinfo = @{} $allpools | foreach { $procinfo[$_] = get-WmiProcess $_ } $procinfo | ConvertTo-Json
Code language: PHP (php)

When ran, you can expect the following output to use in your Zabbix template:

"example.com": { "IOReadOperationsPerSec": 124196, "IOWriteOperationsPerSec": 121311, "IOReadBytesPerSec": 6556014, "WorkingSetPrivate": 109973504, "WorkingSetPeak": 187043840, "HandleCount": 1849, "ThreadCount": 107, "IOWriteBytesPerSec": 68971258, "PercentProcessorTime": 204531250, "WorkingSet": 180256768, "PrivateBytes": 151367680 },
Code language: JavaScript (javascript)

Zabbix template for WMI Win32_PerfRawData_PerfProc_Process counters

In your autodiscover template, you can create an autodiscover item “Fetch”. But first you have to add to your w3wp.conf file the following UserParameter:

UserParameter=Fetch,powershell -NoProfile -ExecutionPolicy Bypass -File C:\path\to\zabbix\scripts\w3wp-wmi.ps1

Now you can use Fetch in your template:

<items> <item> <name>NewFetcher</name> <key>Fetch</key> <history>0</history> <trends>0</trends> <value_type>TEXT</value_type> </item> ...
Code language: HTML, XML (xml)

And a discovery_rule:

<discovery_rules> <discovery_rule> <!-- Yes, I created this for Vevida.com rock solid Windows Server hosting for your websites and applications --> <name>Vevida IIS AppPool Insights</name> <key>w3wp.pooldiscovery</key> <delay>1m</delay> <item_prototypes> <item_prototype> <name>appPool: {#APPPOOLNAME} IOReadOperationsPerSec</name> <type>DEPENDENT</type> <key>iis.apppool.IOReadOperationsPerSec[{#APPPOOLNAME}]</key> <delay>0</delay> <history>7d</history> <value_type>FLOAT</value_type> <description>The rate at which the process is writing bytes to I/O operations. This counter counts all I/O activity generated by the process to include file, network and device I/Os.</description> <application_prototypes> <application_prototype> <name>appPool: {#APPPOOLNAME}</name> </application_prototype> </application_prototypes> <preprocessing> <step> <type>JSONPATH</type> <params>$.IOReadOperationsPerSec</params> </step> <step> <type>CHANGE_PER_SECOND</type> <params/> </step> </preprocessing> <master_item> <key>newfetch[{#APPPOOLNAME}]</key> </master_item> </item_prototype> ... <item_prototype> <name>NewFetch for {#APPPOOLNAME}</name> <type>DEPENDENT</type> <key>newfetch[{#APPPOOLNAME}]</key> <delay>0</delay> <history>0</history> <trends>0</trends> <value_type>TEXT</value_type> <application_prototypes> <application_prototype> <name>appPool: {#APPPOOLNAME}</name> </application_prototype> </application_prototypes> <preprocessing> <step> <type>JSONPATH</type> <params>$['{#APPPOOLNAME}']</params> </step> </preprocessing> <master_item> <key>Fetch</key> </master_item> </item_prototype> </item_prototypes> </discovery_rule> </discovery_rules>
Code language: HTML, XML (xml)

Continue creating an item_prototype for your counter data, and you’re all set. Keep in mind that PercentProcessorTime is measured by the number of CPU seconds a process uses, per second. Meaning 1 is 100%. You can add a multiplier of 100, so 100% is 1 CPU core completely used. For readability, in my template I have two multiplier preprocessing steps: 0.0000001 and 100 (yes, one 0.00001 was also enough). And don’t forget to set CHANGE_PER_SECOND.

Eliminate and discard unsupported monitoring items

Now that you have your template for IIS application pool monitoring complete, what happens to an item if the script doesn’t return a value for -for example- WorkingSetPeak? The item becomes “unsupported”, and you don’t want that. Eliminate Zabbix unsupported items by adding an error_handler to your items as a preprocessing step:

<preprocessing> <step> <type>JSONPATH</type> <params>$.WorkingSetPeak</params> <error_handler>DISCARD_VALUE</error_handler> </step> <step> <type>DISCARD_UNCHANGED_HEARTBEAT</type> <params>10m</params> </step> </preprocessing>
Code language: HTML, XML (xml)

The preprocessing documentation reads:

If you mark the Custom on fail checkbox, the item will not become unsupported in case of failed preprocessing step and it is possible to specify custom error handling options: either to discard the value, set a specified value or set a specified error message.

Are you looking for rock solid, eco-friendly, .NET hosting? Look no further! UmbHost offers powerful hosting services for websites and businesses of all sizes, and is powered by 100% renewable energy!

Windows Server and IIS monitoring with Zabbix, the series

This is part of a series of posts about monitoring Windows Server and IIS (web sites, application pools incluis) with Zabbix. Other editions in this series are:

I hope you like it.

Hi! Join the discussion, leave a reply!

Scroll to Top