Interesting New Software
So for quite some time I have been looking around for a solution to a problem. We are developing a tool in-house at iContact that will allow actual production queries to be run against a database. This actually almost complete. Thursday we actually did the initial testing. I am responsible for setting up a system that will allow a database to be rolled back to a start point. That way we can populate a database — run queries against it and then roll back to re-run the test after configuration changes.
Snapshots with rollback would allow this. There is no common Linux software that allows this. I must admit to a little envy about Solaris as it does allow snapshots with ZFS. I honestly just wish all the developers could get together and figure out some way to port this to Linux. But .. that is another story. I tried working with unionfs. It is just to clunky for what I want to do. It was designed really to work with “live cd” based projects (like Knoppix) and isn’t really suited to my needs.
My CTO pointed me to some software yesterday that I read about some time ago (about two years ago if I remember). I had lost track of it and forgotten about it. The software doesn’t appear to be in active development. The last release was 2-26-06. However, it does appear to do everything I want and then some.
The software is called EVMS (Enterprise Volume Management System) and looks like it was developed in-house at IBM back in 2001. Evidently IBM use to support the project in a fairly big way but it has been somewhat orphaned of late. A few of the developers still work on it but not very much.
The software works on top of underlying filesystem technologies such as LVM and provide a single unified way of working with them. In addition it adds some capabilities of its own (including the afore-mentioned snapshot capabilities). EVMS has a command line interface, a ncurses interface and a gtk+ GUI. Covers it all. Unfortunately the installation looks like it is a little complex. I will be getting with the sys admins Monday to see if I can get some time next week for getting it set up on the test system.
The EVMS website is: http://evms.sourceforge.net
Anyone out there have experience with this software? Would love to hear some feedback.
7 Comments so far
Leave a reply
LVM itself has excellent support for snapshots. The only problem with snapshots (in general) is that the copy-on-write overhead might skew performance results due to a lot of seeking between the data area and snapshot area. I’m assuming you’re doing this for benchmarking.
We already use LVM at work. Yes, LVM will create snapshots. We actually use them to perform backups of the databases. But it doesn’t offer roll-back capabilities (I guess I could point MySQL to the data on the snapshot partition, but that is a hack I really don’t want to do). I want to run test data against the database and then roll it back to the initial state. If we end up using EVMS I am sure it will skew performance somewhat but we will just have to take that into account.
I see. I didn’t read carefully enough.
Why do you want to rollback specifically instead of just having a starting point and getting to that starting point? Is it a speed thing?
Sheeri,
So the idea is that we run production queries captured from other servers. All queries including inserts,updates and deletes. So when the test is done it is necessary to roll the state of the database back to the initial starting point. That way I can make a configuration change to the testing server and have a second run with the exact same set of queries for benchmarking. If I don’t roll back it would be cause problems on the second run. Inserts would fail on duplicate data .. that kind of thing.
Make sense?
In the past, I’ve just used virtual machines to achieve just the same affect. Any reason why that wouldn’t work here?
William,
I am not that familiar with VMs. I work on them more of the level of the sys admins handing me a VM to play with (specifically, I do have for a test/sandbox server but I don’t work with it from the perspective of a system admin at all). I wouldn’t be surprised at all if that were true though. The idea is that with our benchmarking tool I want to run it on a server identical to our production server with production data. Since we don’t run MySQL in VM’s in production that wouldn’t work.
We are done polishing things up. I just have to find the time to write a nice long blog post about the tools as we will be updating the download pages with new versions. I am very pleased with how it turned out.