file io tracing on centos ( part 1)

I'm now working in a CentOS Linux environment, and since my Solaris days are over, I'm feeling a bit handicapped by not having DTrace and iosnoop. It still amazes me that monitoring MyISAM table data and index IO is not more common, especially in heavy duty environments.

My search for an analogous tracing system on Linux lead me to strace. I have been playing with this a little and although I am disappointed overall, I have been able to figure out how to gather what I need. It isn't pretty.

So far, my favored invocation of strace on an active mysqld process is :

strace -q -tt -e trace=open,close,read,write,pread64,pwrite64 -s 256 -f -v -p 2>&1

strace by default directs it's output to STDERR, unless you specify a file ( -o ). I wanted the output to go to STDOUT so I can pipe it into a script that will roll up the output and report by the minute. My goal will be to print an example output something like :
(blogger doesn't usually honor my formatting..not sure how this will look)

time file MbReadMin ReadMin MbWriteMin WriteMin
==== ========================
13:04 /mnt/mysql1/customers.MYI 26 134 0 0
13:04 /mnt/mysql1/sales_summary.MYD 0 0 15 124


This involves a bit of parsing of the output depending on the syscall. The open() syscall gives the actual file name and the "id" , which can be cross referenced with the pread64() and pwrite64() functions, where I am getting the size.

The problem is that you can only get the file name if the table was not open before you invoke strace. And beware, this is a cpu hog.