Filesystem file-creation speed test

Introduction

We recently had issues with file creation times at Better.be. A certain application wrote a lot of different files in a single NFS mounted directory, and it seemed slower than expected. In order to get some real statistics, I wrote a very small C program that writes a configurable number of uniquely named files into the current directory (file-1, file-2, file-3, etc.). It doesn't write any bytes to the file, it merely creates it and closes it. For statistical purposes, it outputs a log file which specifies how many milliseconds it took for each call to fopen and fclose combined. Unfortunately it didn't end up being used (a colleague had written a python application that did the exact same thing), but I decided to have some fun with it anyway.

Linux - Ext3

I ran the application on my Linux box at work, on a local ext3 filesystem. The box is a Quad-Core Intel 2.66 something-ish machine. The results were very interesting.

Files written	Time taken	Average file	Deletion time	Average file
327800	10.4 seconds (10400 ms)	0.0317 ms	3.5 minutes	0.64 ms

I was quite impressed by ext3's file creation speed. Even the 327800th call to fopen and fclose only took about 0.03ms, just like the first one did! However, it appears that 327800 files in the same directory is a hard limit in ext3. Not really an issue for me, although it might be a painful limit for specific server production environments. The one weird and disappoiting thing about ext3 was the time it took to remove the files! A rm -rf on the directory took 3,5 minutes! But still, the entire operation of creating and deleting 300.000+ files took less than 4 minutes. Acceptable in my opinion.

Linux - NFS mount

I then decided to run the same test on my home directory, which is an NFS mount to a file server. It was obvious that it would take longer, simply because of the fact that this time every file creation needed to be communicated over the network to the NFS server. So I expected to see slower times. The results:

Files written	Time taken	Average file	Deletion time	Average file
100000	2 minutes 35 seconds (155000 ms)	1.55 ms	7 minutes 23 seconds (443000 ms)	4.43 ms

No comment. It's way slower than a locally mounted ext3 filesystem, but still it's very, very acceptable for a network mounted directory in my opinion.

Windows XP Home - NTFS

It became really interesting when I decided to run the same application on my home box running Windows XP Home and uses NTFS. It's also a Quad-Core Intel 2.66 something-ish machine. The partition I ran it on is huge, more than 80% free disk space (+/- 250gb drive), so fragmentation wasn't an issue. I didn't really know what to expect... I expected it to be considerably slower, but I wasn't really sure. I started the application and told it to create 500.000 files, and sat back... and waited... and waited... and waited... and waited... while at the start the disk activity led on my case lit up almost non-stop, after 3 hours I saw a single blink every +/- 5 seconds. I was becoming afraid, and decided to kill the process. The results were.... astonishing. It turned out that after 3 hours of work, Windows had only created about 180.000 files! The first few files, fopen and fclose took only about 0.01 ms. But after 3 hours, this had risen to more than 300ms!!! I don't know how Microsoft did it, but they made NTFS unacceptably slow. The only explanation I can think of is that Microsoft has multiple implementations of NTFS: real ones for the most expensive version of Windows, and dumb ones for my XP Home Windows version. I don't know. The results:

Files written	Time taken	Average ms per file	Deletion time	Average file
184020	3 hours (10.800.000 ms)	58.6 ms	15 secs	0.0815 ms

58.6 ms on average. This is outrageous! I really have no words for it. The only very, very positive thing about NTFS is that while creating files is extremely slow, removing them was blazingly fast! It took only 15 seconds for the rmdir command to finish. It's just the way of the Microsoft, I guess ;-)

Oh, one last fun thing. A colleague of mine plotted the time it took for each individual file to create and close. Check out this cool plot (Y-axis is in milliseconds, X-axis is the nth file that got created)

-- Foddex

1 comment(s)

Click to write your own comment