Monit monitoring on Ubuntu 14.04 VM on Hyper-V

In this post you’ll learn about setting up a Monit monitoring service for your websites and services. Monit is a free and open source service monitoring application which can perform various event-based actions. Monit can send email notifications, restart a service or application, or take other responsive actions. We set Monit up on a Ubuntu 14.04 VM, built on Hyper-V. And we use Monit to monitor several websites, and send out notifications on downtime.

Monit monitoring

Monit monitoring for websites and services, on an Ubuntu 14.04 LTS VM, hosted on a Windows Server 2012 R2 or Windows 8.1 Hyper-V.

Monit is a small Open Source utility for managing and monitoring Unix systems. Monit conducts automatic maintenance and repair and can execute meaningful causal actions in error situations.

I recently read about Monit in a DigitalOcean tutorial, and it looked interesting. Interesting enough to start playing around with it. Lacking the extra physical computer, I decided to use a virtual Linux installation in Hyper-V. Even though it’s on Windows 8.1, the same would apply for Windows Server 2012 R2.

Steps

  1. Install Ubuntu Server 14.04.1 LTS VM on Hyper-V
  2. Install Monit and additional services (like Postfix to send out notifications)
  3. Configure Monit (website-, DNS- and SMTP-monitoring at the moment)

This achieves a couple things.

  • Monit is isolated within a VM. Nowadays, anybody can run virtual machines in Hyper-V on their Windows 8.1 or Windows 10 workstation. However, since not everyone has an extra computer to run an additional Linux distribution, we use an Ubuntu VM in Hyper-V. The same applies to Windows Server 2012 R2 Hyper-V of course. (update: now we have Windows Subsystem for Linux, or WSL to host Monit!)

Configure email delivery for Monit, relay through Google Gmail SMTP

First, we have to make sure Monit can send out email notifications. Since we use Monit to monitor websites, we don’t want to use our hosting provider’s SMTP server. That may be down too in case of an event.

We install Postfix and configure Postfix to relay mail through Google Gmail SMTP servers. A Gmail-account is required.

Install all necessary packages:

sudo apt-get install postfix mailutils libsasl2-2 ca-certificates libsasl2-modules

If Postfix Configuration asks for a general type of mail configuration, choose Internet Site. Don’t forget to set a valid FQDN hostname.

Now open your postfix config file:

vi /etc/postfix/main.cf

and add following lines to it:

# SASL SUPPORT FOR SERVERS
#
# The following options set parameters needed by Postfix to enable
# Cyrus-SASL support for authentication of mail servers.
#
relayhost = [smtp.gmail.com]:587
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
smtp_tls_CAfile = /etc/postfix/cacert.pem
smtp_use_tls = yes

We specifiy our Gmail username and password in the file /etc/postfix/sasl_passwd. Create that file using VI:

vi /etc/postfix/sasl_passwd

And add following line:

[smtp.gmail.com]:587    USERNAME@gmail.com:PASSWORD

Fix permission and update Postfix config to use sasl_passwd file:

sudo chmod 400 /etc/postfix/sasl_passwd
sudo postmap /etc/postfix/sasl_passwd

Next, validate certificates to avoid running into error. Just run following command:

cat /etc/ssl/certs/Thawte_Premium_Server_CA.pem | sudo tee -a /etc/postfix/cacert.pem

Finally, reload postfix config for changes to take effect:

sudo /etc/init.d/postfix reload

If all went well, Postfix is now set up to send mail through Google Gmail SMTP servers. This guarantees email delivery when our hosting provider’s SMTP server is unreachable or down. These steps came from a great tutorial on rtcamp.com.

Installing and configuring Monit

Installing Monit on our Ubuntu 14.04 VM is nothing more than:

sudo apt-get install monit

On Ubuntu 14.04, the Monit configuration files are located in /etc/monit/ and the main Monit configuration file is /etc/monit/monitrc. Open up the Monit configuration file:

vi /etc/monit/monitrc

Locate and uncomment the following lines and change them to match your situation:

set mailserver localhost  #Use localhost for email alert delivery.

set mail-format {
      from: monit@$HOST
   subject: monit alert --  $EVENT $SERVICE
   message: $EVENT Service $SERVICE
                 Date:        $DATE
                 Action:      $ACTION
                 Host:        $HOST
                 Description: $DESCRIPTION

            Your faithful employee,
            Monit
}

set alert your_gmail@address.com not on { instance, action }

Let’s restart Monit: service restart monit

Configure Monit to monitor a remote website

The whole point of this tutorial is to set up remote monitoring of websites and services. In the folder /etc/monit/conf.d we can add a configuration file:

sudo vi /etc/monit/conf.d/saotn-org-external

Monitor ICMP response and HTTP connectivity

In the configuration file /etc/monit/conf.d/saotn-org-external we add checks for ICMP responses (ping), and HTTP connectivity:

check host www.saotn.org with address www.saotn.org
  if failed icmp type echo
    for 5 times within 5 cycles
  then alert

# HTTP check
  if failed
    port 80 protocol http
    for 5 times within 5 cycles
  then alert

Configure Monit’s command line interface

If we want to check the status of our monitored websites and services, we need to set up the Monit command line interface. Once again, open up /etc/monit/monitrc, and uncomment the following lines to enable the web service locally:

set httpd port 2812 and
        use address localhost
        allow localhost

Restart Monit to make the configuration current. Now it is possible to check Monit status from the Bash command line. Use sudo monit status to view the current monitoring status:

The Monit daemon 5.6 uptime: 1m

Remote Host 'www.saotn.org'
  status                Online with all services
  monitoring status     Monitored
  icmp response time    0.014s [Echo Request]
  port response time    0.013s to www.saotn.org:80 [DEFAULT via TCP]
  data collected        Sun, 15 Feb 2015 14:58:13

System 'ubuntu-01'
  status                Running
  monitoring status     Monitored
  load average          [0.24] [0.07] [0.06]
  cpu                   0.0%us 0.0%sy 0.0%wa
  memory usage          117772 kB [5.7%]
  swap usage            0 kb [0.0%]
  data collected        Sun, 15 Feb 2015 14:58:13

If needed, you can disable and re-enable Monit monitoring with the commands sudo monit unmonitor all and sudo monit monitor all

Monit monitoring for DNS or SMTP

Monit can be easily used to monitor other remote services as well. Services like DNS or SMTP. To monitor SMTP mail servers, add an smtp-external (or what ever) file to your /etc/monit/conf.d file, and configure the host, address, and alert:

check host smtp-01 with address smtp-01
  if failed port 587 type tcp protocol smtp then alert
check host smtp-02 with address smtp-02
  if failed port 587 type tcp protocol smtp then alert
check host smtp-03 with address smtp-03
  if failed port 587 type tcp protocol smtp then alert

Do the same in your nameservers-external file:

check host dns-01 with address dns-01
  if failed port 53 type udp protocol dns then alert
check host dns-02 with address dns-02
  if failed port 53 type udp protocol dns then alert
check host dns-03 with address dns-03
  if failed port 53 type udp protocol dns then alert

Monit reporting to M/Monit

M/Monit is the reporting service of multiple Monit instances or agents. M/Monit collects the data sent by Monit and displays it in a graphical web interface very neat. Once you have a Monit instance running, setting up M/Monit is as easy as one-two-three. All you literally have to do is following the Setup-guide.

Utilizing the powers of Monit and M/Monit, you can monitor local services/daemons, and restart them when an event happens. You can even monitor remote TCP-services, such as SMTP, DNS or HTTP/HTTPS.

If you want to use Monit in Windows Subsystem for Linux (WSL), then I recommend you read my post Setting up Monit monitoring in Windows Subsystem for Linux WSL.

1 thought on “Monit monitoring on Ubuntu 14.04 VM on Hyper-V”

  1. .toolbelt-social-share{clear:both;font-size:1rem;display:grid;gap:calc(var(–toolbelt-spacing)/ 4);grid-template-columns:repeat(auto-fit,minmax(11rem,1fr));margin-bottom:calc(var(–toolbelt-spacing) * 2);margin-top:calc(var(–toolbelt-spacing) * 2)}.toolbelt-social-share .toolbelt_share-api{display:none}.toolbelt-social-share a{padding:calc(var(–toolbelt-spacing)/ 4) var(–toolbelt-spacing);color:var(–toolbelt-color-light);align-items:center;display:flex;text-decoration:none}.toolbelt-social-share a:hover{color:var(–toolbelt-color-light)}.toolbelt-social-share a:hover span{text-decoration:underline}.toolbelt-social-share-api-enabled .toolbelt-social-share .toolbelt_share-api{display:inline}.toolbelt-social-share-api-enabled .toolbelt-social-share a{display:none}.toolbelt-social-share svg{-webkit-margin-end:calc(var(–toolbelt-spacing)/ 2);margin-inline-end:calc(var(–toolbelt-spacing)/ 2);height:1.5rem;width:1.5rem;vertical-align:middle}.toolbelt-social-share svg *{stroke:none;fill:currentColor}@media (min-width:500px){.toolbelt-social-share .toolbelt_whatsapp{display:none}}
    Are you running into MySQL load problems? Learn how to tune MySQL servers for a heavy InnoDB workload, by configuring innodb_buffer_pool_instances. Dividing the InnoDB buffer pool into multiple instances improves Disk I/O. By doing so, you run your database and website more efficiently and faster. Here is a little help for you.

    (adsbygoogle = window.adsbygoogle || []).push({});

    Tune MySQL InnoDB buffer pool instances & size for heavy workloads

    All for more InnoDB Disk I/O performance on MySQL 5.5+.

    Tuning MySQL servers is an ever ongoing process. Every new MySQL version brings new configuration settings you can use to improve its performance. As a MySQL DBA you want your database server and databases to perform better than well, don’t you?

    MariaDB/MySQL 5.5.4 introduces new configuration settings for the InnoDB storage engine. This can greatly improve MySQL’s InnoDB performance, both in read and write operations.

    One of those settings is innodb_buffer_pool_instances. The innodb_buffer_pool_instances divides the InnoDB buffer pool into separate instances. Dividing your buffer pool into separate instances can improve concurrency, by reducing contention as different threads read and write to cached pages. Multiple buffer pool instances are configured using the innodb_buffer_pool_instances configuration option.

    You might also want to adjust the innodb_buffer_pool_size value:

    The larger the InnoDB buffer pool, the more InnoDB acts like an in-memory database. It reads data from disk once and then accesses the data from memory during subsequent reads. Buffer pool size is configured using the innodb_buffer_pool_size configuration option.

    Back to increasing innodb_buffer_pool_instances.

    The innodb_buffer_pool_instances divides the InnoDB buffer pool in a number of regions.

    The number of regions that the InnoDB buffer pool is divided into. For systems with buffer pools in the multi-gigabyte range, dividing the buffer pool into separate instances can improve concurrency, by reducing contention as different threads read and write to cached pages. Each page that is stored in or read from the buffer pool is assigned to one of the buffer pool instances randomly, using a hashing function. Each buffer pool manages its own free lists, flush lists, LRUs, and all other data structures connected to a buffer pool, and is protected by its own buffer pool mutex.This option takes effect only when you set the innodb_buffer_pool_size to a size of 1 gigabyte or more. The total size you specify is divided among all the buffer pools. For best efficiency, specify a combination of innodb_buffer_pool_instances and innodb_buffer_pool_size so that each buffer pool instance is at least 1 gigabyte.

    In MySQL versions prior to 5.5.4 this was not configurable and thus set to just one instance. Now you can increase innodb_buffer_pool_size, and you can divide the InnoDB buffer pool into multiple regions by setting innodb_buffer_pool_instances to 2, 3, 4 or 8. As long as innodb_buffer_pool_size is set high enough, and you have enough memory available in your MySQL database server. This increases InnoDB read/write threads.

    To enable multiple buffer pool instances, set the innodb_buffer_pool_instances configuration option to a value greater than 1 (the default) up to 64 (the maximum).

    How to restore single MySQL table from a full mysqldump backup file?

    For example, you can set innodb_buffer_pool_size to 6 GB and innodb_buffer_pool_instances to 4 in your my.cnf MySQL configuration file:

    .wp-block-code {
    border: 0;
    padding: 0;
    }

    .wp-block-code > div {
    overflow: auto;
    }

    .shcb-language {
    border: 0;
    clip: rect(1px, 1px, 1px, 1px);
    -webkit-clip-path: inset(50%);
    clip-path: inset(50%);
    height: 1px;
    margin: -1px;
    overflow: hidden;
    padding: 0;
    position: absolute;
    width: 1px;
    word-wrap: normal;
    word-break: normal;
    }

    .hljs {
    box-sizing: border-box;
    }

    .hljs.shcb-code-table {
    display: table;
    width: 100%;
    }

    .hljs.shcb-code-table > .shcb-loc {
    color: inherit;
    display: table-row;
    width: 100%;
    }

    .hljs.shcb-code-table .shcb-loc > span {
    display: table-cell;
    }

    .wp-block-code code.hljs:not(.shcb-wrap-lines) {
    white-space: pre;
    }

    .wp-block-code code.hljs.shcb-wrap-lines {
    white-space: pre-wrap;
    }

    .hljs.shcb-line-numbers {
    border-spacing: 0;
    counter-reset: line;
    }

    .hljs.shcb-line-numbers > .shcb-loc {
    counter-increment: line;
    }

    .hljs.shcb-line-numbers .shcb-loc > span {
    padding-left: 0.75em;
    }

    .hljs.shcb-line-numbers .shcb-loc::before {
    border-right: 1px solid #ddd;
    content: counter(line);
    display: table-cell;
    padding: 0 0.75em;
    text-align: right;
    -webkit-user-select: none;
    -moz-user-select: none;
    -ms-user-select: none;
    user-select: none;
    white-space: nowrap;
    width: 1%;
    }
    ; InnoDB buffer pool size in bytes. The primary value to adjust on a database server,
    ; can be set up to 80% of the total memory in these environments
    innodb_buffer_pool_size = 6000M

    ; If innodb_buffer_pool_size is set to more than 1GB, innodb_buffer_pool_instances

    ; divides the InnoDB buffer pool into this many instances.
    innodb_buffer_pool_instances = 4
    Code language: TOML, also INI (ini)

    In this example, I’ve used an innodb_buffer_pool_size of 6000M (6 GB), so there is 1500M available per innodb_buffer_pool_instance, which is more than the minimum 1 GB. As a rule of thumb, set your innodb_buffer_pool_size to approximately 70 – 80% of the RAM available.

    Innodb_buffer_pool_instances defaults

    Various MySQL versions have different innodb_buffer_pool_instances default values, here is an overview – listing – for you:

    MySQL version# InnoDB buffer pool instancesNotesMySQL 5.5 (<= 5.5.4)1not configurableMySQL 5.51–MySQL 5.6 (<= 5.6.5)1–MySQL 5.6 (>= 5.6.6)8or 1 if innodb_buffer_pool_size < 1GBMySQL 5.78or 1 if innodb_buffer_pool_size < 1GBMariaDB 10 (<= MariaDB 10.0.3)1–MariaDB 10 (>= MariaDB 10.0.4)8–

    InnoDB read and write I/O threads in MySQL

    Besides innodb_buffer_pool_instances, you can also increase the number of InnoDB read I/O threads and write I/O threads. These are configured with innodb_write_io_threads and innodb_read_io_threads.

    Both settings default to 4 threads. We can increase these to, for example, 8:

    ; Number of I/O threads for writes
    innodb_write_io_threads = 8
    ; Number of I/O threads for reads
    innodb_read_io_threads = 8
    Code language: TOML, also INI (ini)

    innodb_read_io_threads

    The number of I/O threads for read operations in InnoDB. The default value is 4.

    innodb_write_io_threads

    The number of I/O threads for write operations in InnoDB. The default value is 4.

    When should you increase the number of innodb_read_io_threads? When you see more than 64 × innodb_read_io_threads pending read requests in SHOW ENGINE INNODB STATUS, you might gain by increasing the value of innodb_read_io_threads.

    Optimizing InnoDB Disk I/O

    If you follow the best practices for database design and the tuning techniques for SQL operations, but your database is still slowed by heavy disk I/O activity, explore these low-level techniques related to disk I/O. If the Unix top tool or the Windows Task Manager shows that the CPU usage percentage with your workload is less than 70%, your workload is probably disk-bound, Optimizing InnoDB Disk I/O.

    Starting from MariaDB 10.0, the default number of innodb_buffer_pool_instances is 8. This means you have to configure your innodb_buffer_pool_size to at least 8 GB, see the defaults above.

    Protip, don’t over optimize: never make too many configuration changes at once. After changing one or two settings, let the server run for a few days so you can learn the effects of the changes. Then, if necessary, make additional changes to the configuration.

    Convert MyISAM to InnoDB tables for WordPress using a pluginFor WordPress, I created a plugin to convert MyISAM tables to InnoDB, that now is incorporated into the Vevida Optimizer WordPress plugin. The Vevida Optimizer plugin extends the automatic update feature already present in WordPress. WordPress core updates can be switched on or off, themes and translations can be automatically updated, and the plugin updates can be configured on a per-plugin basis.

    Innodb_buffer_pool_instances deprecated in MariaDB 10.5.1+

    Please note that in MariaDB 10.5.1 and up, innodb_buffer_pool_instances is deprecated and ignored (MDEV-15058). Oracle MySQL 8.0 still seem to support it though.

    Extra tips for MySQL performance tuning

    Besides optimizing InnoDB for a high-performance workload, there is more you can do to tune MySQL server and database performance. Here are some extra MySQL configuration tips for you. Some information might be outdated and obsolete but may hold valuable information for tuning your MySQL servers. Some settings may be ignored if you’re only using InnoDB as storage engine (tip: drop MyISAM, only use InnoDB!)

    Note: this is a translation and rewrite of my older Dutch post “MySQL performance en optimalisatie tips“, which is now deleted and links to here. Just in case you were wondering why you arrived here instead of the Dutch post after clicking a link :-)

    1: No two MySQL servers are the same

    When optimizing MySQL database servers, keep in mind that no server is equal to another. Settings that work well on one server, may degrade performance on a second. If you manage multiple servers with its configuration under version control (e.g almost -or exactly- the same MySQL configuration for all servers), choose what works best on all servers.

    To determine what you can improve, you first need to know how the server performs now. You can use some MySQL commands for this on your MySQL cli (data comes from my very old post).

    mysql> SHOW STATUS LIKE '%key_read%';
    +-------------------+-------------+
    | Variable_name | Value |
    +-------------------+-------------+
    | Key_read_requests | 11810240259 |
    | Key_reads | 9260357 |
    +-------------------+-------------+
    Code language: SQL (Structured Query Language) (sql)

    These two variables and values relate to the configured key_buffer_size

    In this old example, the database server has 4 GB of RAM and a configured key_buffer_size of 512 MB. The ratio (Key_read_requests / Key_reads) is approximately 1/1275, which is good but the key_buffer_size value may be increased to 768 MB. Even though this is not yet necessary.

    mysql> SHOW STATUS LIKE 'thread%';
    +-------------------+---------+
    | Variable_name | Value |
    +-------------------+---------+
    | Threads_cached | 0 |
    | Threads_connected | 76 |
    | Threads_created | 6234040 |
    | Threads_running | 2 |
    +-------------------+---------+
    Code language: SQL (Structured Query Language) (sql)

    These Threads_* variable values show you there are currently 76 connected threads, of which only 2 are really running a thread (executing a statement). This means 74 connections are idle.

    Here you can also see that there is no “thread cache” set up for MySQL: Threads_cached | 0

    You can use the MySQL Server System variable thread_cache_size to configure how many threads must be cached by MySQL. This is one of those configuration settings that, probably, provides the least performance gain, but still…

    Don’t set this one too high, somewhere between 20 and 40 is often good enough:

    thread_cache_size = 20Code language: TOML, also INI (ini)

    When you execute the previous statement again, the values will be:

    mysql> SHOW STATUS LIKE 'thread%';
    +-------------------+-------+
    | Variable_name | Value |
    +-------------------+-------+
    | Threads_cached | 14 |
    | Threads_connected | 98 |
    | Threads_created | 2896 |
    | Threads_running | 1 |
    +-------------------+-------+
    Code language: SQL (Structured Query Language) (sql)

    You now have 14 threads cached :)

    2: Miscellaneous MySQL configuration settings

    A few words on some miscellaneous configuration settings.

    2.1: tmp_table_size and max_heap_table_sizeThe default tmp_table_size and max_heap_table_size values are 16M. These two have to be equal in size! It sets the maximum size for internal in-memory tables, resulting in less creation of temporarily MyISAM tables on the file system. That in return, results in less disk I/O.

    https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_tmp_table_sizehttps://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_max_heap_table_size

    2.2: join_buffer_sizeThe join_buffer_size sets a maximum buffer size for plain index scans, range index scans and joins without indices (and therefore perform full table scans). Keep this one low, 1M for example.

    MySQL database optimization with indices

    3. Use Diagnostics for improvements

    It is important to frequently run diagnostics and/or look up diagnostic data (for example in your information_scheme table). Percona has a lot of information about some key metrics:

    https://www.percona.com/doc/percona-server/8.0/index.html#diagnostics-improvements

    MySQL tuning, the conclusion

    Tuning MySQL and the InnoDB storage engine is an important step in further optimizing your hosting environment. Every new MySQL version brings new settings to improve your MySQL configuration, so be sure to read those changelogs.

    In this article we went over InnoDB Buffer Pool Size and InnoDB Buffer Pool Instances. Setting these properly greatly improves your MySQL server’s performance!

    But never (ever, ever) over-optimize! Please don’t make too many configuration changes at once. Make one or two and restart mysqld. After monitoring your system for a few days, running with the new configuration, you have data available to further optimize other MySQL settings.

    With InnoDB being the default storage engine, you also have to make sure you make use of this storage engine in MySQL. Therefore it is important to convert old MyISAM tables to InnoDB.
    Related PostsHow to string replace on all WordPress posts in MySQLHow to compare MD5 and SHA1 hashes in MySQLMySQL database optimization with indicesRestore single MySQL table from a full mysqldump backup fileShare this:Click to share on Twitter (Opens in new window)Click to share on Facebook (Opens in new window)Click to share on Pocket (Opens in new window)Click to share on LinkedIn (Opens in new window)Click to share on WhatsApp (Opens in new window)Click to share on Reddit (Opens in new window)Click to share on Tumblr (Opens in new window)Click to share on Pinterest (Opens in new window)Click to share on Telegram (Opens in new window)Click to share on Skype (Opens in new window)Click to email this to a friend (Opens in new window)Click to print (Opens in new window)var toolbelt_social_share_description = “Are you running into MySQL load problems? Learn how to tune MySQL servers for a heavy InnoDB workload, by configuring…”;Share Tweet this
    Share this
    Share this
    Save this
    Share this

Comments are closed.