Archive for August, 2010

XFS versus JFS…

August 18, 2010

I recently built a new linux dev box and was faced with a choice I’d made several times before.  What file system do I want to use.  A little background on the hardware I’m using – it’s a quad core machine with plenty of ram and for storage I’m using 3 7200 RPM 1TB Hitachi drives (Deskstar 7K1000.C – SATA 3Gb/s) in a software RAID.  I have the boot partition mirrored on all 3 drives, I have a swap partition on each drive as well (all mounted with equal priority) and then the rest of the drive is a RAID 5 LVM partition.  Within the LVM partition, I’ve created the root and /home filesystems.

I’d heard that XFS offered some interesting features compared to JFS as far as taking snapshots of the filesystem state for backups, etc so I decided to give it a try.  I’d seen some benchmarks that showed that XFS was slower on metadata writes, but hadn’t fully appreciated what that might mean in my day to day usage and how drastic the difference could be.

Well here is one benchmark that I found as I was building the system that I couldn’t believe, and has made me since switch back to JFS.

When I was extracting the linux kernel source and the Gentoo stage 3 tar it seemed like it was taking a longer time that usual to extract.  Afterwards, I decided to do a test to compare XFS to JFS (which is what I’ve used on my other machines), so I created another LVM partition, formatted it JFS and did a quick and dirty comparison.

Copying the the linux kernel source tree (with the compiled object files in it – 716MB) on JFS (copying to/from the same partition)

time (cp -a linux-2.6.34-gentoo-r1 test && sync)
real    2m28.284s
user    0m0.377s
sys     0m4.525s

Versus copying it on XFS (again to/from the same partition)

time (cp -a linux-2.6.34-gentoo-r1 test && sync)
real    6m56.095s
user    0m0.372s
sys     0m5.076s

And the deletes show even worse performance


time (rm -rf test && sync)
real    0m27.679s
user    0m0.043s
sys     0m0.818s


time (rm -rf test && sync)
real    4m47.683s
user    0m0.054s
sys     0m1.842s

I even tried some XFS mount options that are supposed to optimize it for RAID5 reads and writes (the sunit and swidth) options

Since I’m doing software development on this box, I’m dealing with lots of small source file operations, so I’m guessing that XFS would have been considerably slower for my work load.  I use the IntelliJ Idea IDE for my Java development, and I’ve read blog posts saying that it’s performance is sensitive to disk performance (I’m guessing for all the caches, etc they create, plus scanning for changes in the sources every time you switch in and out of the IDE – I’ve never tried it on an SSD to see how that compares).

When comparing filesystems benchmarks I’d found previously, I’d always assumed that the difference in performance in actual day to day usage wouldn’t be very dramatic between the different systems.  Sure maybe one was 30% faster – not a huge penalty if you’re getting handy features, but 2-3x difference in actual (fairly common) usage floored me.  Your mileage may vary, and it could be that I have something configured incorrectly with the RAID that causes XFS to perform this poorly compared to JFS.