Getting more out your Windows Performance Counters monitoring for web applications – part 3

IIS AppPool Insights in Zabbix – because there is always more than one way

This is part 3 about Zabbix monitoring for your websites and ASP.NET applications in IIS. This time I’ll show you how to get data from Win32_PerfRawData_PerfProc_Process counter, fast, for everey application pool This counter is notorious for its slowness, but you can get data a bit faster.

In part 1 I showed you how to create an IIS website autodiscovery item and application discovery, and in part 2 you monitor the performance of your ASP.NET web application utilizing Windows performance counters in Zabbix. Now we take it a step further, to monitor Win32_PerfRawData_perfProc_Process data for all currently running application pools (autodiscovered).

Protip: I query PerfRawData, because using Win32_PerfFormattedData_perfProc_Process costs a small performance hit, and results may be rounded to 0 and not 0,1.

Autodiscover running IIS application pools

In this part you’ll query Win32_PerfRawData_PerfProc_Process performance counters using WMI/CIM in PowerShell. The Win32_PerfRawData_PerfProc_Process class contains a lot of usefull properties for your IIS website application pools.

The gathered data needs to be transformed into a JSON blob so Zabbix can use it, but you also need to somehow match your application pool’s (w3wp.exe) process ID to the result of your WMI query (called “WQL”, or SQL for WMI. See Querying with WQL, WQL (SQL for WMI) for more information). This is done in PowerShell.

Let’s begin.

Create a file called w3wp-discovery.ps1 and add the following into it:

$mypools = Get-CimInstance -EA SilentlyContinue -Query "select AppPoolName,ProcessId from WorkerProcess" -Namespace root\webadministration | Select @{Name='{#APPPOOLNAME}';Expression={$_.AppPoolName}},ProcessId
Code language: PowerShell (powershell)

This will return an object $mypools with all discovered IIS application pools and a macro {#APPPOOLNAME}. It looks like:

{#APPPOOLNAME} ProcessId -------------- --------- 7652 5156
Code language: PowerShell (powershell)

The macro {#APPPOOLNAME} is used to reference application pools in your autodiscover template. We need this information later on.

Zabbix conf

For Zabbix, create a configuration file w3wp.conf in your zabbix_agentd.confd folder, and add the following UserParameter to it:

UserParameter=w3wp.discovery,powershell -NoProfile -ExecutionPolicy Bypass -File C:\path\to\w3wp-discovery.ps1
Code language: PowerShell (powershell)

This creates the w3wp.discovery key.

Counter query Powershell script for dependent items

To query Win32_PerfRawData_perfProc_Process for performance counter values, we need PowerShell. You can use Get-WmiObject of Get-CimInstance cmdlets, and the data I want to monitor for application pools is (you are free to choose your own of course).

  • HandleCount
  • IDProcess
  • IOReadBytesPerSec
  • IOReadOperationsPerSec
  • IOWriteBytesPerSec
  • IOWriteOperationsPerSec
  • PercentProcessorTime
  • PrivateBytes
  • ThreadCount
  • WorkingSet
  • WorkingSetPeak

This is where the magic happens and you get all that info:

$processinfo = Get-CimInstance -EA SilentlyContinue -Query "select * from Win32_PerfRawData_PerfProc_Process where Name like 'w3wp_%'" -Namespace root\cimv2 | Select HandleCount,IDProcess,IOReadBytesPerSec,IOReadOperationsPerSec,IOWriteBytesPerSec,IOWriteOperationsPerSec,PercentProcessorTime,PrivateBytes,ThreadCount,WorkingSet,WorkingSetPeak
Code language: PowerShell (powershell)

The WQL queries Win32_PerfRawData_PerfProc_Process for all data, and then I use PowerShell Select to filter the output. Why? Because this is way faster than running a query per application pool like "select * from Win32_PerfRawData_PerfProc_Process where IDProcess='w3wp.exe PID'". Especially if you need to this 300 or 500 times on a server. WMI is really slow, so let PowerShell do the filtering of data.

The output may look as follows:

HandleCount : 2537 IDProcess : 7652 IOReadBytesPerSec : 123547753 IOReadOperationsPerSec : 112674 IOWriteBytesPerSec : 78788008 IOWriteOperationsPerSec : 88060 PercentProcessorTime : 4461250000 PrivateBytes : 278888448 ThreadCount : 159 WorkingSet : 283172864 WorkingSetPeak : 288657408 HandleCount : 3038 IDProcess : 5156 IOReadBytesPerSec : 27230185458 IOReadOperationsPerSec : 3777431 IOWriteBytesPerSec : 87141280 IOWriteOperationsPerSec : 89040 PercentProcessorTime : 248751718750 PrivateBytes : 1265516544 ThreadCount : 183 WorkingSet : 1267384320 WorkingSetPeak : 1358626816
Code language: PowerShell (powershell)

It doesn’t look like something you can work with in a Zabbix template, but fortunately you can use PowerShell to transform this in a nicely formatted JSON. This is why we need the IDProcess property; it get’s matched to an application pool’s .ProcessId property.

Note: I decided to write my own JSON output, and not to use PowerShell’s ConvertTo-Json cmdlet. I just couldn’t get the output right for Zabbix.

The following funcion get’s called in a ForEach loop to select the correct data for each ProcessId / IDProcess.

function getWmiProcess($wpname) { Try { $data = ($processinfo | Where-Object IDProcess -eq $($mypools | Where-Object '{#APPPOOLNAME}' -eq "${wpname}").ProcessId) If($data) { $computerdata = [pscustomobject][ordered]@{ {"IOReadOperationsPerSec"} = "$($data.IOReadOperationsPerSec)" + "," {"IOWriteOperationsPerSec"} = "$($data.IOWriteOperationsPerSec)" + "," {"IOReadBytesPerSec"} = "$($data.IOReadBytesPerSec)" + "," {"WorkingSetPeak"} = "$($data.WorkingSetPeak)" + "," {"HandleCount"} = "$($data.HandleCount)" + "," {"ThreadCount"} = "$($data.ThreadCount)" + "," {"IOWriteBytesPerSec"} = "$($data.IOWriteBytesPerSec)" + "," {"PercentProcessorTime"} = "$($data.PercentProcessorTime)" + "," {"WorkingSet"} = "$($data.WorkingSet)" + "," {"PrivateBytes"} = "$($data.PrivateBytes)" } return $computerdata } } Catch { } }
Code language: PowerShell (powershell)

And the ForEach loop to get the data and eventually output the JSON:

$temp = @() $temp += "`{`r" ForEach ($item in $mypools."{#APPPOOLNAME}") { $temp += "`"${item}`": `{`n" $temp += getWmiProcess $item if($item -eq $mypools."{#APPPOOLNAME}"[-1]) { $temp += "`}" } else { $temp += "`}`," } } $temp += "}" $temp
Code language: PowerShell (powershell)

When ran, you can expect the following output to use in your Zabbix template:

{ "": { "IOReadOperationsPerSec" : 112674, "IOWriteOperationsPerSec" : 88060, "IOReadBytesPerSec" : 123547753, "WorkingSetPeak" : 288657408, "HandleCount" : 2537, "ThreadCount" : 159, "IOWriteBytesPerSec" : 78788008, "PercentProcessorTime" : 4461250000, "WorkingSet" : 283172864, "PrivateBytes" : 278888448 }, "": { "IOReadOperationsPerSec" : 3777431, "IOWriteOperationsPerSec" : 89040, "IOReadBytesPerSec" : 27230185458, "WorkingSetPeak" : 1358626816, "HandleCount" : 3038, "ThreadCount" : 183, "IOWriteBytesPerSec" : 87141280, "PercentProcessorTime" : 248751718750, "WorkingSet" : 1267384320, "PrivateBytes" : 1265516544 }, }
Code language: PowerShell (powershell)

Zabbix template for WMI Win32_PerfRawData_PerfProc_Process counters

In your autodiscover template, you can create an autodiscover item “Fetch”. But first you have to add to your w3wp.conf file the following UserParameter:

UserParameter=Fetch,powershell -NoProfile -ExecutionPolicy Bypass -File C:\path\to\zabbix\scripts\w3wp-wmi.ps1
Code language: PowerShell (powershell)

Now you can use Fetch in your template:

<items> <item> <name>NewFetcher</name> <key>Fetch</key> <history>0</history> <trends>0</trends> <value_type>TEXT</value_type> </item> ...
Code language: HTML, XML (xml)

And a discovery_rule:

<discovery_rules> <discovery_rule> <!-- Yes, I created this for rock solid Windows Server hosting for your websites and applications --> <name>Vevida IIS AppPool Insights</name> <key>w3wp.pooldiscovery</key> <delay>1m</delay> <item_prototypes> <item_prototype> <name>appPool: {#APPPOOLNAME} IOReadOperationsPerSec</name> <type>DEPENDENT</type> <key>iis.apppool.IOReadOperationsPerSec[{#APPPOOLNAME}]</key> <delay>0</delay> <history>7d</history> <value_type>FLOAT</value_type> <description>The rate at which the process is writing bytes to I/O operations. This counter counts all I/O activity generated by the process to include file, network and device I/Os.</description> <application_prototypes> <application_prototype> <name>appPool: {#APPPOOLNAME}</name> </application_prototype> </application_prototypes> <preprocessing> <step> <type>JSONPATH</type> <params>$.IOReadOperationsPerSec</params> </step> <step> <type>CHANGE_PER_SECOND</type> <params/> </step> </preprocessing> <master_item> <key>newfetch[{#APPPOOLNAME}]</key> </master_item> </item_prototype> ... <item_prototype> <name>NewFetch for {#APPPOOLNAME}</name> <type>DEPENDENT</type> <key>newfetch[{#APPPOOLNAME}]</key> <delay>0</delay> <history>0</history> <trends>0</trends> <value_type>TEXT</value_type> <application_prototypes> <application_prototype> <name>appPool: {#APPPOOLNAME}</name> </application_prototype> </application_prototypes> <preprocessing> <step> <type>JSONPATH</type> <params>$['{#APPPOOLNAME}']</params> </step> </preprocessing> <master_item> <key>Fetch</key> </master_item> </item_prototype> </item_prototypes> </discovery_rule> </discovery_rules>
Code language: HTML, XML (xml)

Continue creating an item_prototype for your counter data, and you’re all set.

Windows Server and IIS monitoring with Zabbix, the series

This was part 2 of a series of posts about monitoring Windows Server and IIS (web sites, application pools incluis) with Zabbix. Other editions in this series are:

I hope you like it.

Sharing is caring
Show Buttons
Hide Buttons