0

Are live virtual machine backups capable of not corrupting complex databases such as MySQL? Is the memory map included in virtual machine backups, so that ephemeral data such as disk caches and unsaved changes are preserved, and a restored virtual machine essentially resumes its prior powered-on state?

Or do virtual machine backups still require additional separate backup procedures, such as periodically writing a backup of the live database to a static backup file kept inside the virtual machine?

The specific scenario I am thinking of is live backups of virtual machines on VMware using Veeam. However I am requesting this also in general for other hypervisors.

2
  • It would depend on the platform. Hyper-V production checkpoints specifically exclude memory. If a file is tightly coupled to memory it would not work. Also snapshots are not backups. A backup from within the guest should always work properly. Commented Feb 10 at 8:32
  • Also it appears that MySQL recommends flushing the tables with read lock before snapshot as part of the process. dev.mysql.com/doc/refman/8.4/en/… Commented Feb 10 at 9:52

2 Answers 2

1

A DBMS that can be recovered from a crash, can be backed up by copying the files online. Crash recovery is desirable for data you care about, because abnormal terminations sometimes happen, virtual machine or not.

Often this is implemented with journals or logs. File system replays its log on mount, so your file system is at least not corrupt. DBMS replays its log, so the internal physical structure is at least valid.

During a scheduled backup, we can be more polite and make things easier for ourselves. Taking a consistent backup of data files may involve:

  • Pausing DBMS writes. Not issuing any more pwrite() system calls or similar allows in flight I/O to finish.
  • Switching the log, as a convenient start point of this backup.
  • Taking a copy of all the data files at the same point in time. LVM snapshot, storage array clone, file system snapshot. File copy likely takes too long, and is not one point in time. Multiple LUNs or file systems is tricky, really it should be one exact point in time for consistency.
  • Resuming DBMS writes.
  • Copying the snapshot to protection storage. As the database is online this is less time sensitive. Although still needs to be completed to meet recovery objectives.

Obviously the devil is in the details.

Veeam MySQL examples have a "HotBackup Database Freeze" scenario, where the write pause uses FLUSH TABLES WITH READ LOCK. The data copy is VM snapshot based. Resuming writes happens when a post script terminates the lock holder of the pre script.

PostgreSQL backup and restore docs describe file system level backup in general. There is not an equivalent write pause. For convenience, you can CHECKPOINT to set a point in time where all the data files have been written. Also, if enough logs are archived, consistent file backups are not required, and recovery to arbitrary points in time is possible.

As always, whatever the implementation, actually restore occasionally to prove it works.

0

They definitely would have been true for any local database, if you include memory snapshot into the backup. I don't know a generic virtualization backup technology that does that.

Without this, next best backup procedure includes snapshotting all the storage at the exact same instant of time and storing the snapshot. It would be seen as a sudden power reset by the OS if you restore the VM from that backup. Therefore, any database that can withstand a sudden machine reset will be properly recovered with such backup technique.

So here it depends on the database. Any fully ACID database engine, such as PostgreSQL, Oracle, or MySQL's InnoDB, will survive the backup, because it's exactly what "D" in the acronym means, durability. Notice that it isn't guaranteed for other MySQL's engines, for example, I am not sure MyISAM is ACID.


The problem with such backup for the database is not that it is insufficient in terms of storing the database contents. It is not the reason why it is not employed often for the most important databases. One of the real problems is, this way of backup procedure puts more load to the system, because it needs to basically read all the database disk images every time you backup, which strains storage, which is already a weak point of a loaded database. Another is that it is also very demanding to use: you have to provide the exact same resources if you wish to restore from the backup and that procedure is also quite consuming, both in terms of time and load.

Mind you, not every restore from the backup is performed to recover from the total loss of the system. In most cases, you need to retrieve a few bits of the backup, such as a few WAL segments/archive logs (that's the same thing named differently in different databases), and it's a waste to recover and spin the entire VM just for that. Some advanced backup systems (Proxmox Backup when paired with Proxmox VE) can partially alleviate the problem since they can parse filesystems in the backup image and retrieve individual files and also run the VM early while it is still being recovered from the backup, but I wouldn't recommend depending on that possibility.

More straightforward backup techniques such as only backing up these WALs all the time and only doing full backups not very often incur a less IO load on the system and require less backup storage, but also have other compelling advantages, such as possibility of a point in time recovery, possibility to efficiently combine backup with HA (completed WALs can be copied to the standby systems and to backup simultaneously, which is the basis for the most HA database setups, everything else is built on top of that). On the other hand, backups by exporting databases are more useful, because they allow you to spin the database on entirely another system, recover individual tables and so on, while being also less consuming.

So, overall it depends on the purpose of the VM and the database in it, but for many cases, consider backing up the database individually, as recommended by the database vendor, in addition to backing up the VM image infrequently.

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.