Comparison of NTFS drivers for Mac OS X

In this article I will compare the three current NTFS drivers for Mac OS X Lion as well as give a little definition about filesystems and about the major ones that can be used for interoperatibility between Windows and Mac OS X.

What's a filesystem?

A filesystem is the way that files are organized in a disk. Every major operating system comes with its own one, incompatible with the others. Windows comes with NTFS, Mac OS X comes with HFS+ and Linux with ext4.

This articles centers on comparing how to interchange files between Windows and Mac OS X so only the filesystems that can be commonly used by both will be discussed. Those are NTFS, FAT32, exFAT and HFS+.

There is also another filesystem, UDF (from Universal Disk Format) that was created to be universally compatible with all operating systems. It's the one used by DVD and Bluray discs, but its usage outside this kind of disks (like for example in hard disks or flash drives) requires deep knowledge of the operating system, as for example, Windows won't allow to read a UDF formatted hard disk unless tricked to do so, and Mac OS X doesn't offer an option to format anything this way easily.

What's FAT32?

FAT32 (also known as File Allocation Table, 32 bit) is an evolution of the original MS-DOS filesystem, introduced in 1995 with Windows 95 OSR2. It's a fragile filesystem with severe limitations (maximum filesize is 4Gb, Windows cannot format drives bigger than 32Gb on it, it does not support user security/permissions, fragments heavily, etc) but however almost every operating system out there can read and write it perfectly. It's also used extensively in any kind of device using mass storage (phones, digital cameras, DVRs, etc).

What's NTFS?

NTFS (also known as NT File System) is the native filesystem introduced with Windows NT. It's a very clever filesystem, that introduces compression, alternate data streams, security, etc. However its specification are a secret, and just only in the recent years enough of it has been reverse engineered to allow drivers to fully support it.

What's HFS+?

HFS+ (also known as Hierarchical File System Plus and Mac OS Extended) is the native Mac OS X filesystem, first introduced with Mac OS 8.1. It supports features unique to Mac OS like FinderInfo and the Resource Fork. It added new features from UNIX (permissions, links) and alternate data streams, security, etc, over its predecesor, HFS.

What's exFAT?

exFAT (also known as Extended File Allocation Table) is a new filesystem designed for Microsoft to be used on flash drives. It overcomes the most important FAT32 limitations without the complexity of NTFS, and is mandatory to any device claiming to support the SDXC and Memory Stick XC cards to support it. However, its specification is secret, patented and require a paid license to implement a driver (I still need confirmation from a lawyer that this applies in the EU). Currently only Mac OS X and Windows Vista or 7 support it out-of-the-box.

A little filesystem terms glossary

Alternate Data Stream (aka Fork): An ADS or Fork is a piece of data that comes alongside the file's data, but is independent of it. It should not be confused with metadata, that can be stored inside the data, and can be stored as ADSs. It can contain things like image thumbnails, URL where the file was downloaded from, the instant messaging contact's name who send it, so on.

Resource Fork: The Resource Fork is a special ADS that Mac OS previously depended extensively on (not anymore on Mac OS X, but files from previous versions and its applications can be simply broken if they lose it). It's specifically designed as a database of data types, like application pieces, rich text style, so on. NTFS stores it as an ADS called "AFP_Resource", and HFS+ stores it independently. Mac OS X stores them in filesystems that don't support it in separate files prepending "._" to the filename.

Extended Attributes (aka EA or xattr): Extended attributes are small alternate data streams usually limited in size. Its concept was first introduced in OS/2 to store things like the file icon, the filename, etc. FAT versions precious to FAT32 supports them natively (only on OS/2 and Windows NT), NTFS stores them as ADSs and Mac OS X 10.4 calls its ADSs so (but it doesn't limit the size). They are also supported by most Linux filesystems.

Finder Info: Finder Info is a collection of attributes that were required by Mac OS versions previous to Mac OS X. They store the icon positions in a folder, provide support for the Aliases, and stores which application created the file and what kind of file is it (Mac OS previous to X required this to know that a JPEG is a JPEG and it should be opened with Photoshop, X can live without this). NTFS stores it as an ADS called "AFP_AfpInfo".

Alias: The Alias is a Mac OS feature, where it creates a shortcut to a file that works even if the file is still moved to another place (it stores the file and volume unique identifiers to always find the file wherever it is). They are implemented using the resource fork, and OS/2 provided a similar feature called Shadow using EAs.

Symbolic link: A UNIX feature, a file is created that shorcuts to the exact location of another file, so if you move the original file the link is broken.

Hard link: A UNIX feature where a file information is duplicated but not its content. That's, there are two files that are really just the same (you modify one, both get modified), and until all hard links of the same file are not deleted the file isn't either.

The tests, explained

In this article I'll test the three NTFS offerings for Mac OS X that are currently available and supported under Mac OS X Lion, as well as the exFAT, FAT32 and HFS+ filesystems, for features and speed.

While Mac OS X Lion comes with an NTFS driver of itself, designed in-house by Apple, opensource and programmed by the guy that did most of the NTFS reverse engineering, it's as of today, unsupported. Drives are not writeable unless you do some command line tricks (that I will not explain in this article), and even if you do this nothing is warranted to work. If it makes your hard disk burn, or you lose all your data, or an alien abducts you for using it, Apple will say "we told you it's not supported". However, it passed all my tests without a single corruption, so, at your own risk!!!

Paragon Software offers us, among other filesystem drivers, with Paragon NTFS for Mac, compatible with Lion from 9.0, however they do some things I don't like. They exaggerated their features comparison saying that the Apple driver doesn't support NTFS features it does however supported even before Paragon did.

Finally Tuxera offers us Tuxera NTFS for Mac, a commercial solution that evolved from the opensource NTFS-3G widely used for NTFS support on Linux. They still support PowerPC macs (my PowerBook G4 is happy) and were the first to support storing the Resource Fork how NTFS is designed to store them, inside an ADS and not a "._" file.

For the test I basically did two things. The first was creating a separate folder inside the volume, copying different files. One is a Mac OS 7 application, that heavily relies on its Resource Fork (it's data being 0 bytes indeed) and its Finder Info. Then I copied a disk image file that contains four different Mac OS X extended attributes. Then I created some plain text files with names that are not supported by Windows (but they are supported by NTFS, see below). Then I created a symbolic link of a JPEG file and a hard link of it.

Finally for the speed tests I used Bonnie++, with this command line: bonnie++ -d /Volumes/Test/test/ -s 4095 -n 100 -m fat32 -u claunia -f -b

Test machine is a MacBookPro5,4 with 4Gb of DDR1066 RAM, Mac OS X Lion 10.7.2 (11C26), and a Samsung 2.5" IDE hard disk inside an Alcor Micro USB 2.0 case. Tests were done with all applications closed and through a network shell so the GUI is in background and not consuming important foreground memory and cycles.

Note on filenames

NTFS supports ANY character in a filename but the NULL one (byte 0x00). Windows however does not support characters from 0x00 to 0x31, "<", ">", ":", " " " (no spaces, double quotes character), "/", "\", "|", "?" and "*". It also doesn't support files that are end on a space or a dot. More information here. While those files can exist in NTFS volumes, no Windows application will ever be able to do anything with them (access, rename, move, copy, delete, open, close, say it, won't work).

UNIX system supports ANY but NULL and "/".

Mac OS supports ANY but NULL and ":". Mac OS X stores files containing "/" as ":" in HFS+ but shows it to you as "/" (tricked!). Also Mac OS includes the Apple logo character "" (you won't see it unless you're reading this article under any Mac OS version), that's unsupported anywhere else.

Microsoft designed Windows NT to be interoperable in networks containing Mac OS clients, and to serve files to them (with the now deprecated Services for Macintosh product), so they created a way to store filenames unsupported by Windows on NTFS, so Windows will be able to access ithem, and Mac OS will be able also, showing the correct filename to Mac OS network clients, and a fake one to Windows machines. It's described here.

Feature comparison

	HFS+	Apple's NTFS	Paragon NTFS	Tuxera NTFS	FAT32	exFAT
Resource fork	Supported	Supported	Partial	Supported	Partial	Partial
Extended attributes	Supported	Supported	Partial	Supported	Partial	Partial
FinderInfo	Supported	Supported	Partial	Supported	Partial	Partial
Hard links	Supported	Supported	Supported	Supported	Unsupported	Unsupported
Symbolic links	Supported	Partial	Supported	Supported	Unsupported	Unsupported
Filenames	Supported	Supported	Supported	Supported	Supported	Unsupported
Cost	Free	Free	$19.95 / version	$36.25	Free	Free

Partial support means it's stored in "._" files, supported that it's stored as the filesystem designed them to be stored, and unsupported that they simply, won't work.

On links there is a curious difference between the three NTFS implementations.

The Tuxera symbolinc link seems to use the POSIX interface of NTFS, and it's not directly accepted by Windows explorer. However, it's the way NTFS was designed to. Paragon does the same. And Services for UNIX will recognize it as a symbolic link.

Apple's implementation however stores it a a simple file containing the path to the destination, and neither Windows explorer or Services for UNIX recognize it as a link.

Tests

So the Bonnie++ tests outputs shows the following.

Version 1.93c			Sequential Output						Sequential Input				Random Seeks			Sequential Create						Random Create
Concurrency		Size	Per Char		Block		Rewrite		Per Char		Block				Num Files	Create		Read		Delete		Create		Read		Delete
			K/sec	% CPU	K/sec	% CPU	K/sec	% CPU	K/sec	% CPU	K/sec	% CPU	/sec	% CPU		/sec	% CPU	/sec	% CPU	/sec	% CPU	/sec	% CPU	/sec	% CPU	/sec	% CPU
HFS+	1	4095M			24722	4	11657	2			35292	2	274.3	3	100	18243	77	44878	99	9321	58	253	2	52737	99	107	1
HFS+	Latency				344ms		518ms				35023us		257ms		Latency	99931us		1438us		16793us		858ms		1187us		1066ms
Apple's NTFS	1	4095M			24663	6	12276	2			35382	3	288.9	17	100	401	60	1558	22	839	16	327	68	172	2	71	1
Apple's NTFS	Latency				425ms		619ms				58872us		291ms		Latency	99838us		73278us		346ms		468ms		46196us		692ms
Paragon NTFS	1	4095M			24760	4	11675	2			35400	3	295.0	5	100	206	81	3203	33	455	12	186	77	275	3	42	1
Paragon NTFS	Latency				296ms		530ms				22544us		161ms		Latency	587ms		36047us		119ms		424ms		76842us		596ms
Tuxera NTFS	1	4095M			20470	5	11094	6			31946	4	170.2	4	100	410	68	3875	18	1120	15	95	19	22	0	12	0
Tuxera NTFS	Latency				180ms		382ms				193ms		4658ms		Latency	79166us		138ms		1395ms		36988ms		37577ms		848ms
FAT32	1	4095M			24342	2	11708	2			35293	3	264.3	7	100	56	89	440	93	119	47	69	87	619	90	79	43
FAT32	Latency				528ms		586ms				74162us		3436ms		Latency	286ms		34503us		343ms		368ms		20362us		440ms
exFAT	1	4095M			24665	2	11305	2			35095	3	254.2	23	100	51	79	305	85	69	53	49	79	399	80	56	50
exFAT	Latency				363ms		514ms				262ms		1456ms		Latency	323ms		69381us		333ms		356ms		55743us		443ms

Lots of data to interpret so let's put it in pretty graphs.

This is the bare speed test. Here all the filesystems work quite at the same speed, with a little slowdown on Tuxera NTFS when writing or reading files. Not too much to discuss it, they are all as fast so it's just your choice of flavour.

Things get different when you're treating with folders filled with files. When you copy a folder full of files (say, the iTunes folder, your photo library, so on) you're creating it. When you open it in the Finder, you're reading it. And when you empty the trash you're deleting it.

So this matters more than a single file speed, shown in previous test, because you will do these operations more commonly.

Clearly, HFS+ comes to a huge advantage here.

Of the three NTFS offerings, Apple's one clearly needs more optimization, and Tuxera's gets the lead, being the fastest when multiple file operations are done. Nowhere near of HFS+ however.

FAT32 and exFAT clearly show their design flaws here, being desperately slow in multiple files operation.

Conclusion

This is not a choice that can be simply left to speed tests, it's a complex matter.

If you have a Mac computer, you have bootcamp, and it includes HFS+ drivers for Windows, so clearly this should be the choice isn't it?

However, your friends, university, college, so on, probably won't, cannot, don't want, to have this driver installed. So you're forced to choose between one of the natively supported by Windows filesystems (NTFS, exFAT or FAT32).

FAT32 is not an option, it's not just slow, you will never be able to store any file bigger than 4Gb, that's it, forget about it, simply, erase from your mind, let it be only used by your digital camera and bye bye.

exFAT may seem the future, however it's just an evolution of FAT32 (supports files bigger than 4Gb, but anyway, same internal design), but it's still slow. However it's free to use on Mac OS X and if you don't care about the annoying "._" files that will be created all the way, just use it.

On NTFS, clearly Apple should end it and make it easy to use, integrated, free, writeable. Ok, we've been waiting years for this to come, so sit and wait, and on the meantime decide between Paragon and Tuxera.

Tuxera seems more expensive (but you pay just once per all versions in the future), slower on single files, but handles multiple files faster, and stores things the way they should be stored.

Paragon seems cheaper (you end paying more as every release you pay again), faster on single files, slower on multiple files, and creates the annoying "._" files everywhere.

So Tuxera and Paragon are quite equalled, it's up to you which one to chose.

Personally, I chose Tuxera, because I want to have the confidence that what I write on a NTFS drive using Mac OS X will be correctly identified by Windows applications as what they are (not only Services for Macintosh, Windows explorer recognize AFP_* ADS and inform you will lose them when moving to a filesystem that does not support them) without the annoying "._" files.