Diamond Notes

Just another WordPress weblog

Query Processing Tools Update

I just wanted to take a minute and let people know that we are continuing to work internally on the query processing tools. The querysniffer tool is feature complete for now (and I can’t imagine what else you might need with it). It grabs the MySQL data from the network and injects it into a database table storing the query, the time stored, the total execution time of the query and the host it was captured from. In addition, you can tell it to not grab the data modifying statements (inserts, updates and deletes). Also, it can replace varibles (”WHERE username=”Jack”) with something like this: WHERE username={string}. Why is this useful? So you can group the queries together. Even though one query might be searching for Jack and another query might be searching for Jill, they are both the same query. Can’t think of any other options, but I probably missed one or two.

After all that work, it becomes simple to write a web interface of php or perl to display statistics. I already have one set up internally displaying the most executed queries and the average execution and total execution time fo a query. While I will release this as part of the package, this will be something that each DBA will end up spending some time tweaking to fit their own needs I am sure.

I haven’t really started working on the benchmarking part of the toolkit yet. I have an idea of implementing a unionfs(??) so that I can run the queries against a database and then roll the database back without to much difficulty. That way I can run INSERTs, UPDATEs and DELETEs without worrying about.

We have some internal paperwork to take care of before these new additions can be released but I am confident it won’t be long until that is taken care of. As soon as it is done we will release the new code to the world.

2 Comments so far

  1. Jan Kneschke October 5th, 2007 1:12 pm

    This is pretty much what the MySQL Proxy does, just in a pass-through way. See

    http://jan.kneschke.de/2007/9/14/mysql-proxy-adaptive-slow-query-log

    You may want to add scripting with lua too to make the sniffer extensible for the users.

  2. bmurphy October 5th, 2007 2:06 pm

    Jan,

    Believe it or not we started working on the query tool the week before I saw your first public announcement about mysql-proxy. Yes, at least most of this functionality is in mysql-proxy. The query processing toolkit continues to evolve though and will probably end up meeting some different needs.

    thanks!!

    Keith

Leave a reply