Monitor .NET CLR Garbage Collected heap from your web application

Photo of author
Written By Jan Reilink

Windows Server system administrator & enthusiast.

Are you worried about your .NET webapp running out of memory? In order to let your .NET (web) application run smooth over a longer period of time, it is important to monitor the .NET CLR Garbage Collector (GC) and collection. The what? The .NET Common Language Runtime Garbage Collector. Here is how you can monitor this in a Zabbix template using Powershell WMI / CIM and Win32_PerfRawData_NETFramework_NETCLRMemory Windows Performance Counters.

Microsoft explains the fundamentals of garbage collection, memory allocation and release. As either a .NET developer or systems administrator, we’re all too familiar with – as I call it – memory pollution. Suddenly your .NET web application throws an “OutOfMemoryException”, and you have no idea why. Remember The Dangers of the Large Object Heap? Monitor your .NET web application’s memory usage in Zabbix to understand where memory goes.

Here is how to create Zabbix monitoring for Win32_PerfRawData_NETFramework_NETCLRMemory in your template. For example, monitor the “Large Object Heap Size” performance counter to gain insight in how many bytes are allocated to the large object heap. The same for Gen0PromotedBytesPerSec and Gen1PromotedBytesPerSec performance counters. These counters display the bytes per second that are promoted from generation 0 (youngest) to generation 1, and generation 1 to generation 2 (oldest). Memory is promoted when it survives a garbage collection. You can collect all this information using WMI and PowerShell.

Many of the steps to follow are the same as they are in part 3 of this series, see the link above. Therefore I’ll provide you with the relevants to query Windows Win32_PerfRawData_NETFramework_NETCLRMemory Performance Counters and you can repeat the earlier steps.

Get all application pools by querying WMI root\webadministration namespace

This queries Windows Management Instrumentation (WMI) root\webadministration namespace for all application pools and saves into an object $allpools.

$allpools = ( Get-CimInstance -EA SilentlyContinue -ClassName applicationpool -Namespace root\webadministration ).Name
Code language: PowerShell (powershell)

You do need to put this in one line.

Query WMI for all application pool’s process id’s

You also need to have the process id belonging to an application pool, query the same root\webadministration namespace:

$allprocids = ( Get-CimInstance -EA SilentlyContinue -ClassName WorkerProcess -Namespace root\webadministration ) | Select AppPoolName,ProcessId
Code language: PowerShell (powershell)

Process queried Win32_PerfRawData_NETFramework_NETCLRMemory properties

This is (part 1 of) where the real magic happens. I’ll post the code for you first:

function get-WmiNetClrMem($wpname) { Try { $procid = ($allprocids | Where-Object AppPoolName -eq "${wpname}").ProcessId $data = ($netclrmemory | Where-Object ProcessID -eq $procid) $hashtable = @{ AllocatedBytesPersec = $data.AllocatedBytesPersec Gen0heapsize = $data.Gen0heapsize Gen0PromotedBytesPerSec = $data.Gen0PromotedBytesPerSec Gen1heapsize = $data.Gen1heapsize Gen1PromotedBytesPerSec = $data.Gen1PromotedBytesPerSec Gen2heapsize = $data.Gen2heapsize LargeObjectHeapsize = $data.LargeObjectHeapsize NumberTotalcommittedBytes = $data.NumberTotalcommittedBytes NumberTotalreservedBytes = $data.NumberTotalreservedBytes PercentTimeinGC = $data.PercentTimeinGC } @($hashtable.keys) | % { if (-not $hashtable[$_]) { $hashtable.Remove($_) } } return $hashtable } Catch { } }
Code language: PowerShell (powershell)

A lot is going on here:

  • The ProcessId property for the application pool currently queried is selected from all Process ID’s:
    • $procid = ($allprocids | Where-Object AppPoolName -eq "${wpname}").ProcessId
  • That $procid property then is used to match a ProcessID in the object $netclrmemory (we’ll come to that one later), and is saved in object $data:
    • $data = ($netclrmemory | Where-Object ProcessID -eq $procid)
  • All WMI properties we want are filtered out the $data object and saved into a hashtable
  • The hashtable is stripped from empty values; ConvertTo-Json returns the JSON representation of null when the input object is $null. Zabbix then sees this as an string, which fails when you’re expecting an integer.
  • The hashtable is returned, having valid values for
    • AllocatedBytesPersec
    • Gen0heapsize
    • Gen0PromotedBytesPerSec
    • Gen1heapsize
    • Gen1PromotedBytesPerSec
    • Gen2heapsize
    • LargeObjectHeapsize
    • NumberTotalcommittedBytes
    • NumberTotalreservedBytes
    • PercentTimeinGC

Or less if a property didn’t have a value.

Now for part 2 of the magic.

Query WMI for all required Win32_PerfRawData_NETFramework_NETCLRMemory properties

This is where you actually query Windows Management Instrumentation (WMI) / CIM Win32_PerfRawData_NetFramework_NETCLRMemory for properties you want to monitor in Zabbix. Be creative.

$netclrmemory = (Get-CimInstance -EA SilentlyContinue -ClassName Win32_PerfRawData_NETFramework_NETCLRMemory -Filter "Name like 'w3wp%'") | select AllocatedBytesPersec,Gen0heapsize,Gen1heapsize,Gen2heapsize,LargeObjectHeapsize,NumberTotalcommittedBytes,NumberTotalreservedBytes,PercentTimeinGC,ProcessID,Gen0PromotedBytesPerSec,Gen1PromotedBytesPerSec $netclrinfo = @{} $allpools | foreach { $netclrinfo[$_] = get-WmiNetClrMem $_ } $netclrinfo | ConvertTo-Json
Code language: PowerShell (powershell)

Using Get-CimInstance, you query the aforementioned WMI Class, and filter on the process name like 'w3wp%', since you are only interested in IIS application pools – or worker processes. Output is saved in an object $netclrmemory, the one you noticed above.

A new hashtable object $netclrinfo is created, and you loop through $allpools in a foreach, executing the function get-WmiNetClrMem($wpname). The end result in hashtable $netclrinfo is then converted into an JSON string for Zabbix.

Here is the JSON output:

"reilink.nl(domain)(4.0)(pool)": { "LargeObjectHeapsize": 74864, "Gen0heapsize": 1660248, "Gen2heapsize": 2805364, "Gen1heapsize": 341388, "NumberTotalreservedBytes": 33546240, "NumberTotalcommittedBytes": 7282688, "PercentTimeinGC": 48, "Gen0PromotedBytesPerSec": 114060, "Gen1PromotedBytesPerSec": 0, "AllocatedBytesPersec": 80405312 },
Code language: JSON / JSON with Comments (json)

Here you can see 114060 bytes per second is promoted from Generation 0 garbage collection to Generation 1 (“Gen0PromotedBytesPerSec”). As Microsoft explains: Objects that are not reclaimed in a garbage collection are known as survivors and are promoted to the next generation.

Add the following XML to your template file:

<item_prototype> <name>.NET CLR Memory: Gen0heapsize</name> <type>DEPENDENT</type> <key>iis.apppool.Gen0heapsize[{#APPPOOLNAME}]</key> <delay>0</delay> <history>7d</history> <description>This counter displays the maximum bytes that can be allocated in generation 0 (Gen 0); its does not indicate the current number of bytes allocated in Gen 0. A Gen 0 GC is triggered when the allocations since the last GC exceed this size. The Gen 0 size is tuned by the Garbage Collector and can change during the execution of the application. At the end of a Gen 0 collection the size of the Gen 0 heap is infact 0 bytes; this counter displays the size (in bytes) of allocations that would trigger the next Gen 0 GC. This counter is updated at the end of a GC; its not updated on every allocation.</description> <units>B</units> <application_prototypes> <application_prototype> <name>{#APPPOOLNAME}</name> </application_prototype> </application_prototypes> <preprocessing> <step> <type>JSONPATH</type> <params>$.Gen0heapsize</params> <error_handler>DISCARD_VALUE</error_handler> </step> <step> <type>DISCARD_UNCHANGED_HEARTBEAT</type> <params>10m</params> </step> </preprocessing> <master_item> <key>netclrmemfetch[{#APPPOOLNAME}]</key> </master_item> </item_prototype>
Code language: HTML, XML (xml)

Repeat for other metrics, and you’re all done!

Here are three graphs as an example:

.NET CLR Memory - AllocatedBytesPerSec
.NET CLR Memory – AllocatedBytesPerSec
.NET CLR Memory - Gen0PromotedBytesPerSec
.NET CLR Memory – Gen0PromotedBytesPerSec
.NET CLR Memory - LargeObjectHeapsize
.NET CLR Memory – LargeObjectHeapsize

Are you looking for rock solid, eco-friendly, .NET hosting? Look no further! UmbHost offers powerful hosting services for websites and businesses of all sizes, and is powered by 100% renewable energy!

Windows Server and IIS monitoring with Zabbix, the series

This is part of a series of posts about monitoring Windows Server and IIS (web sites, application pools incluis) with Zabbix. Other editions in this series are:

I hope you like it.


Did you like: Monitor .NET CLR Garbage Collected heap from your web application

Then please, take a second to support Sysadmins of the North and donate!

Your generosity helps pay for the ongoing costs associated with running this website like coffee, hosting services, library mirrors, domain renewals, time for article research, and coffee, just to name a few.



1 thought on “Monitor .NET CLR Garbage Collected heap from your web application”

Hi! Join the discussion, leave a reply!