Comments on: XFS Snapshots with LVM2

By: k-gee

k-gee — Wed, 20 Feb 2008 09:29:33 +0000

lvm2 kind of knows about xfs and thus applies the xfs_freeze command automatically …. therefore it hangs if you try to apply it by hand.

By: admin

admin — Sat, 19 Jan 2008 01:26:31 +0000

I haven’t had time to really look into this more (other than insuring my lvm backups weren’t done off xfs partitions). However, I had another interesting experience just yesterday. I had upgrade a server and went to resync it off the master. I realized when I went to do it that the master data partition was xfs. Since I had already started I went ahead and did the transfer and started it (the slave) up. Surprisingly it worked.

I had tried the same thing on the previous set of servers at least four times without success. It took a relatively long time to start up, but it did. I don’t have any explanation at this point. My only guess might be some hardware difference causing a problem on the first setup.

By: Aaron

Aaron — Sat, 19 Jan 2008 00:48:11 +0000

Does anyone have more information about LVM “knowing about” XFS? I’m currently running XFS on Suse 10.1 with LVM2 version 2.02. I must be using an older version than people on here because I don’t see the lvcreate hanging behavior. In fact, I have to run xfs_freeze before lvcreate or I end up with corrupted tables. I’d really appreciate it if someone could point me towards some more in depth information with regard to XFS and LVM2.

Thanks,
-Aaron

By: admin

admin — Mon, 31 Dec 2007 04:45:11 +0000

Kristian,

I have used the mylvmbackup tool before. I didn’t in this case but you are right — it just duplicates the steps you outlined. Those steps are exactly what I did except I didn’t do step one. I started with the ‘flush tables with read lock’. All other steps where the same. And executing the ‘flush tables’ first, if I understand, just makes the ‘flush tables with read lock’ execute faster.

So, I still don’t know why it happens.

By: Kristian Köhntopp

Kristian Köhntopp — Sun, 30 Dec 2007 22:50:47 +0000

There is a tool “mylvmbackup” at http://lenz.homelinux.org/mylvmbackup/ written by MySQL’s Lenz Grimmer. It will do everything for you.

The procedure basically is

1. FLUSH TABLES. Force MyISAM buffers to disk as far as possible. Does not lock.
2. FLUSH TABLES WITH READ LOCK. Force MyISAM buffers to disk that have been dirtied since the last FLUSH TABLES. This time, all writeres are stalled.
3. lvcreate -s the snap.
4. UNLOCK TABLES, writers resume.

5. Mount the snap, using -o nouuid in the case of XFS.
6. tar and rsync the snap away.
7. Destroy the snap.

Optionally, a step 5b is performed, starting a secondary server on a different port for InnoDB log recovery before backup. This code currently is slightly broken.

The xfs_freeze is no longer necessary with LVM2. LVM2 and XFS know of each other and will do just fine. As you observed, it does not even work any longer. That is precisely because of the XFS lock around the lvcreate -s.

When moving InnoDB it is very important to get the actual error message from the log. It is also very important to have the my.cnf file with the exact original InnoDB tablespace parameters and match it to the copy. InnoDB will not start if the tablespace declarations of the log size declarations are botched. The actual exact error message from the log will tell you.

An lvcreate -s of an InnoDB is recording InnoDB in a “crashed” state – when you mount and copy that InnoDB, it will be reported as crashed and will do a Redo and an Undo log recovery. That is normal and not a problem – FLUSH TABLES WITH READ LOCK does not flush the dirty blocks from the InnoDB buffer pool. It will however force out all outstanding Redo and Undo log writes, so the snapshot will be complete and after log recovery also consistent. It is just that the log recovery will need to be performed. A wait is not necessary.

By: admin

admin — Sun, 30 Dec 2007 01:30:52 +0000

Partha,

I am aware of the parameters to use with mounting. So you just issue a ‘flush tables with read lock’ on a running server and wait for some period of time..then do the lvm snapshot? Or are you doing something else? If so, please outline as there seem to be a number of people who have run into this..

thanks,

keith

By: Partha Dutta

Partha Dutta — Sat, 29 Dec 2007 17:33:35 +0000

We also use XFS for our reporting server farm for InnoDB. We routinely rsync the filesystem to other servers using LVM snapshots without problems.

What we have found out is that we wait till the innodb buffer pool finishes flushing to disk before creating the snapshot. Also, when mounting the snapshot, we have to mount it a special way, else LVM runs into issues. For example: mount -onouuid,ro /dev/sysvg/mysql2 /inno2

With this, we have flawless snapshoting and rsync capabilities. Hope this helps.

Partha

By: William

William — Fri, 28 Dec 2007 15:40:59 +0000

Thanks for the tip. I was thinking about migrating to xfs the for our new servers. I think that will have to be shelved for now.

By: Michael Moody

Michael Moody — Fri, 28 Dec 2007 09:02:37 +0000

Just a mention here, we were looking for decent mysql backup solutions for a while, and while the lvm method works fine, we also stumbled across a company called r1soft (r1soft.com) that makes a fairly good hot backup software, working on the same principle as the lvm method.

Let me know if you get the xfs/lvm2 snapshot working, I’d be very interested to know how.

Michael