Enabling debug mode logging in KVM

There’s another approach to logging in KVM: configuring debug logging. In the libvirtd configuration file that we just mentioned, there are additional settings you can use to configure this very option. So, if we scroll down to the Logging controls part, these are the settings that we can work with:

  • log_level
  • log_filters
  • log_outputs

Let’s explain them step by step. The first option – log_level – describes log verbosity. This option has been deprecated since libvirt version 4.4.0. In the Logging controls section of the file, there’s additional documentation hardcoded into the file to make things easier. For this specific option, this is what the documentation says:

Figure 16.1 – Logging controls in libvirtd.conf

Figure 16.1 – Logging controls in libvirtd.conf

What people usually do is see the first part of this output (Logging level description), go to the last line (Iog_level), set it to 1, save, restart the libvirtd service, and be done with it. The problem is the text part in-between. It specifically says that journald does rate limiting so that it doesn’t get hammered with logs from one service only and instructs us to use the log_filters setting instead.

Let’s do that, then – let’s use log_filters. A bit lower in the configuration file, there’s a section that looks like this:

Figure 16.2 – Logging filters options in libvirtd.conf

Figure 16.2 – Logging filters options in libvirtd.conf

This gives us various options we can use to set different logging options per object types, which is great. It gives us options to increase the verbosity of things that we’re interested in at a desired level, while keeping the verbosity of other object types to a minimum. What we need to do is remove the comment part of the last line (#log_filters="1:qemu 1:libvirt 4:object 4:json 4:event 1:util" should become log_filters="1:qemu 1:libvirt 4:object 4:json 4:event 1:util") and configure its settings so that they match our requirements.

The third option relates to where we want our debug logging output file to be:

Figure 16.3 – Logging outputs options in libvirtd.conf

Figure 16.3 – Logging outputs options in libvirtd.conf

Important Note

After changing any of these settings, we need to make sure that we restart the libvirtd service by typing in the systemctl restart libvirtd command.

If we’re only interested in client logs, we need to set an environment variable called LIBVIRT_LOG_OUTPUTS to something like this (let’s say we want DEBUG-level logging):

export LIBVIRT_LOG_OUTPUTS="1:file:/var/log/libvirt_guests.log"

All these options are valid until the next libvirtd service restart, which is quite handy for permanent settings. However, there’s a runtime option that we can use when we need to debug a bit on the fly, without resorting to permanent configuration. That’s why we have a command called virt-admin. We can use it to set our own settings. For example, let’s see how we can use it to get our current settings, and then how to use it to set temporary settings:

Figure 16.4 – Runtime libvirtd debugging options

Figure 16.4 – Runtime libvirtd debugging options

We can also delete these settings by issuing the following command:

virt-admin daemon-log-filters ""

This is something that’s definitely recommended after we’re done debugging. We don’t want to use our log space for nothing.

In terms of straight-up debugging virtual machines – apart from these logging options – we can also use serial console emulation to hook up to the virtual machine console. This is something that we’d do if we can’t get access to a virtual machine in any other way, especially if we’re not using a GUI in our environments, which is often the case in production environments. Accessing the console can be done as follows:

virsh console kvm_domain_name

In the preceding command, kvm_domain_name is the name of the virtual machine that we want to connect to via the serial console.

Related Articles

How to add swap space on Ubuntu 21.04 Operating System

How to add swap space on Ubuntu 21.04 Operating System

The swap space is a unique space on the disk that is used by the system when Physical RAM is full. When a Linux machine runout the RAM it use swap space to move inactive pages from RAM. Swap space can be created into Linux system in two ways, one we can create a...

read more

Lorem ipsum dolor sit amet consectetur


Submit a Comment

Your email address will not be published. Required fields are marked *

seventeen − sixteen =