Securing your Zabbix MySQL database

Another great added feature for the Zabbix server is the ability to encrypt data between the database and Zabbix components. This is particularly useful when you are running a split database and the Zabbix server over the network. A Man in the Middle (MITM) or other attacks can be executed on the network to gain access to your monitoring data.

In this recipe, we’ll set up MySQL encryption between Zabbix components and the database to add another layer of security.

Getting ready

We are going to need a Zabbix setup that uses an external database. I’ll be using the Linux lar-book-secure-db and lar-book-secure-zbx hosts.

The new server called lar-book-secure-zbx will be used to connect externally to the lar-book-secure-db database server. The database servers won’t run our Zabbix server; this process will run on lar-book-secure-zbx.

Make sure that MariaDB is already installed on the lar-book-secure-db host and that you are running a recent supported version that is able to use encryption.

How to do it…

  1. Make sure your host files on both hosts from the Getting ready section contain the hostname and IP for your Linux hosts and edit the file with the following:
    vim /etc/hosts
  2. Then, fill in the file with your hostnames and IPs. It will look like this:
    10.16.16.170 lar-book-secure-db
    10.16.16.171 lar-book-secure-zbx
  3. On the lar-book-secure-db MySQL server, if you haven’t already, create the Zabbix database by logging in to MySQL:
    mysql -u root -p
  4. Then, issue the following command to create the database:
    create database zabbix character set utf8 collate utf8_bin;
  5. Also, make sure to create a user that will be able to access the database securely. Make sure the IP matches the IP from the Zabbix server (and Zabbix frontend if seperated):
    create user 'zabbix'@'%' identified BY 'password';
    grant all privileges on zabbix.* to 'zabbix'@*';
    flush privileges;
  6. Quit MySQL and then make sure to run the secure mysql script with the following:
    mysql_secure_installation
  7. Log in to lar-book-secure-zbx and install the Zabbix server repo with the following command:
    rpm -Uvh https://repo.zabbix.com/zabbix/5.0/rhel/8/x86_64/zabbix-release-5.0-1.el8.noarch.rpm dnf clean all
  8. Then, install the Zabbix server and its required components.

    Use the following RHEL-based command:

    dnf install zabbix-server-mysql zabbix-web-mysql zabbix-apache-conf zabbix-agent2 mariadb

    Use the following Debian-based command:

    apt install zabbix-server-mysql zabbix-frontend-php zabbix-apache-conf zabbix-agent mariadb
  9. From the Zabbix server, connect to the remote database server and import the database scheme with the following command:
    zcat /usr/share/doc/zabbix-server-mysql*/create.sql.gz | mysql -h 10.16.16.170 -uzabbix -p zabbix
  10. Now we are going to open the file called openssl.cnf and edit it by issuing the following command:
    vim /etc/pki/tls/openssl.cnf
  11. In this file, we need to edit the following lines:
    countryName_default             = XX
    stateOrProvinceName_default     = Default Province
    localityName_default            = Default City
    0.organizationName_default      = Default Company Ltd
    organizationalUnitName_default. = 

    It will look like this filled out completely:

    Figure 11.12 – OpenSSL config file with our personal defaults

    Figure 11.12 – OpenSSL config file with our personal defaults

  12. We can also see this line:
    dir     = /etc/pki/CA      # Where everything is kept
  13. This means the default directory is /etc/pki/CA; if yours is different, act accordingly. Close the file by saving and continue.
  14. Let’s create a new folder for our private certificates using the following command:
    mkdir -p /etc/pki/CA/private
  15. Now, let’s create our key pair in the new folder. Issue the following command:
    openssl req -new -x509 -keyout /etc/pki/CA/private/cakey.pem -out /etc/pki/CA/cacert.pem -days 3650 -newkey rsa:4096
  16. You will be prompted for a password now:
    Figure 11.13 – Certificate generation response asking for a password

    Figure 11.13 – Certificate generation response asking for a password

  17. You might also be promoted to enter some information about your company. It will use the default we filled in earlier, so you can just press Enter up until Common Name.
  18. Fill in Root CA for Common Name and add your email address like this:
    Figure 11.14 – Certificate generation response asking for information, Root CA

    Figure 11.14 – Certificate generation response asking for information, Root CA

  19. Next up is creating the actual signed certificates that our Zabbix server will use. Let’s make sure that OpenSSL has the right files to keep track of signed certificates:
    touch /etc/pki/CA/index.txt
    echo 01 > /etc/pki/CA/serial
  20. Then, create the folders to keep our certificates in:
    mkdir /etc/pki/CA/unsigned
    mkdir /etc/pki/CA/newcerts
    mkdir /etc/pki/CA/certs
  21. Now, let’s create our certificate signing request for the lar-book-secure-zbx Zabbix server with the following command:
    openssl req -nodes -new -keyout /etc/pki/CA/private/zbx-srv_key.pem -out /etc/pki/CA/unsigned/zbx-srv_req.pem -newkey rsa:2048
  22. You will be prompted to add a password and your company information again. Use the default up until Common Name. We will fill out our Common Name, which will be the server hostname, and we’ll add our email address like this:
    Figure 11.15 – Certificate generation response asking for information, lar-book-secure-zbx

    Figure 11.15 – Certificate generation response asking for information, lar-book-secure-zbx

  23. Let’s do the same for our lar-book-secure-db server:
    openssl req -nodes -new -keyout /etc/pki/CA/private/mysql-srv_key.pem -out /etc/pki/CA/unsigned/mysql-srv_req.pem -newkey rsa:2048

    The response will look like this:

    Figure 11.16 – Certificate generation response asking for information, lar-book-secure-db

    Figure 11.16 – Certificate generation response asking for information, lar-book-secure-db

    Important note

    Our certificates need to be created without a password; otherwise, our MariaDB and Zabbix applications won’t be able to use them. Make sure to specify the -nodes option.

  24. Now, sign the certificate for lar-book-secure-zbx with the following command:
    openssl ca -policy policy_anything -days 365 -out /etc/pki/CA/certs/zbx-srv_crt.pem -infiles /etc/pki/CA/unsigned/zbx-srv_req.pem
  25. You will be promoted with the question Sign the certificate? [y/n]. Answer this and all the following questions with Y.
  26. Now, let’s do the same thing for the lar-book-secure-db certificate:
    openssl ca -policy policy_anything -days 365 -out /etc/pki/CA/certs/mysql-srv_crt.pem -infiles /etc/pki/CA/unsigned/mysql-srv_req.pem
  27. Let’s log in to the lar-book-secure-db MySQL server and create a directory for our newly created certificates:
    mkdir /etc/my.cnf.d/certificates/
  28. Add the right permissions to the folder:
    chown -R mysql. /etc/my.cnf.d/certificates/
  29. Now, back at the new lar-book-secure-zbx Zabbix server, copy over the files to the database server with the following commands:
    scp /etc/pki/CA/private/mysql-srv_key.pem root@10.16.16.170:/etc/my.cnf.d/certificates/mysql-srv.key
    scp /etc/pki/CA/certs/mysql-srv_crt.pem root@10.16.16.170:/etc/my.cnf.d/certificates/mysql-srv.crt
    scp /etc/pki/CA/cacert.pem root@10.16.16.170:/etc/my.cnf.d/certificates/cacert.crt
  30. Now, back at the lar-book-secure-db MySQL server, add the right permissions to the files:
    chown -R mysql:mysql /etc/my.cnf.d/certificates/
    chmod 400 /etc/my.cnf.d/certificates/mysql-srv.key
    chmod 444 /etc/my.cnf.d/certificates/mysql-srv.crt
    chmod 444 /etc/my.cnf.d/certificates/cacert.pem
  31. Edit the MariaDB configuration file with the following command:
    vim /etc/my.cnf.d/mariadb-server.cnf
  32. Add the following lines to the configuration file under the [mysqld] block:
    bind-address=lar-book-secure-db
    ssl-ca=/etc/my.cnf.d/certificates/cacert.crt
    ssl-cert=/etc/my.cnf.d/certificates/mysql-srv.crt
    ssl-key=/etc/my.cnf.d/certificates/mysql-srv.key
  33. Log in to MySQL with the following command:
    mysql -u root -p
  34. Make sure our Zabbix MySQL user requires SSL encryption with the following:
    alter user 'zabbix'@'10.16.16.171' require ssl;
    flush privileges;
  35. Quit and then restart MariaDB with the following command:
    systemctl restart mariadb
  36. Now, back on the lar-book-secure-zbx Zabbix server, create a new folder for our certificates:
    mkdir -p /var/lib/zabbix/ssl/
  37. Copy the certificates over to this folder with the following:
    cp /etc/pki/CA/cacert.pem /var/lib/zabbix/ssl/
    cp /etc/pki/CA/certs/zbx-srv_crt.pem /var/lib/zabbix/ssl/zbx-srv.crt
    cp /etc/pki/CA/private/zbx-srv_key.pem /var/lib/zabbix/ssl/zbx-srv.key
  38. Edit the Zabbix server configuration file to use these certificates:
    vim /etc/zabbix/zabbix_server.conf
  39. Make sure the following lines match our lar-book-secure-db database server’s setup:
    DBHost=lar-book-secure-db
    DBName=zabbix
    DBUser=zabbix
    DBPassword=password
  40. Now, make sure our SSL-related configuration matches our new files:
    DBTLSConnect=verify_full
    DBTLSCAFile=/var/lib/zabbix/ssl/cacert.pem
    DBTLSCertFile=/var/lib/zabbix/ssl/zbx-srv.crt
    DBTLSKeyFile=/var/lib/zabbix/ssl/zbx-srv.key
  41. Also, make sure to add the right permissions to the SSL-related files:
    chown -R zabbix:zabbix /var/lib/zabbix/ssl/
    chmod 400 /var/lib/zabbix/ssl/zbx-srv.key
    chmod 444 /var/lib/zabbix/ssl/zbx-srv.crt
    chmod 444 /var/lib/zabbix/ssl/cacert.pem
  42. Before restarting, make sure to edit the PHP timezone with the following command:
    vim /etc/php-fpm.d/zabbix.conf
  43. Edit the following line to match your timezone:
    ; php_value[date.timezone] = Europe/Riga
  44. Start and enable the Zabbix server with the following:
    systemctl restart zabbix-server zabbix-agent2 httpd php-fpm
    systemctl enable zabbix-server zabbix-agent2 httpd php-fpm
  45. Then, navigate to the Zabbix frontend and fill in the right information as shown in the following screenshot:
    Figure 11.17 – Zabbix frontend configuration, database step

    Figure 11.17 – Zabbix frontend configuration, database step

  46. When we press Next step, we need to fill out some more information:
    Figure 11.18 – Zabbix frontend configuration, server details step

    Figure 11.18 – Zabbix frontend configuration, server details step

  47. Then, after clicking Next step, Next step, and Finish, the frontend should now be configured and working.

How it works…

This was quite a long recipe, so let’s break it down quickly:

  • In Steps 1 through 9, we prepared our servers
  • In Steps 10 through 37, we executed everything needed to create our certificates
  • In Steps 38 through 47, we set up our Zabbix frontend for encryption

Going through all these steps, setting up your Zabbix database securely can seem like quite a daunting task, and it can be. Certificates, login procedures, loads of settings, and more can all add up to become very complicated, which is why I’d always recommend diving deeper into encryption methods before trying to set this up yourself.

If your setup requires encryption, though, this recipe is a solid starting point for your first-time setup. It works very well in an internal setting, as we are using private certificates. Make sure to renew them yearly, as they are only valid for 365 days.

All Zabbix components, except for communication between the Zabbix server and Zabbix frontend, can be encrypted as shown in the following diagram:

Figure 11.19 – Zabbix encryption scheme possibilites

Figure 11.19 – Zabbix encryption scheme possibilites

We’ve set up encryption between the following:

  • The Zabbix server and MariaDB
  • The Zabbix frontend and MariaDB

This means that when our Zabbix server or frontend requests or writes data to our database, it will be encrypted. Because our Zabbix applications are running on a different server than our Zabbix database, this might be important. For example, our setup might look like this:

Figure 11.20 – Zabbix setup with external network diagram

Figure 11.20 – Zabbix setup with external network diagram

Let’s say the cloud is called Some company in a network that isn’t managed by us. There are several switches and routers in this network that are used for numerous clients with their own VLANs. If one of these devices gets compromised somehow, all of our Zabbix data could be seen by others.

Even if the network equipment is ours, there might still be a compromised device in the network and our data can be seen. This is why you might want to add encryption, to add that extra layer of security. Whether it’s against breaches in other companies and their network that you want to secure against or whether it’s against your own breaches, securing your database as we did in this recipe might just save you from leaking all that data.

Related Articles

How to add swap space on Ubuntu 21.04 Operating System

How to add swap space on Ubuntu 21.04 Operating System

The swap space is a unique space on the disk that is used by the system when Physical RAM is full. When a Linux machine runout the RAM it use swap space to move inactive pages from RAM. Swap space can be created into Linux system in two ways, one we can create a...

read more

Lorem ipsum dolor sit amet consectetur

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *

3 + fifteen =