APFS: New Apple File System Changes Everything

August 9th, 2016

APFS Apple File System

APFS, or Apple File System, is one of the biggest changes coming to every new Apple device beginning next year. Backing up your files is our job, so we think a new file system is very interesting. Let’s take a look at APFS to understand what it is and why it’s so important.

File systems are a vital component of any computer or electronic device. The file system tells the computer how to interact with data. Whether it’s a picture you’ve taken on your phone, a Microsoft Word document, or an invisible file the computer needs, the file system accounts for all of that stuff.

File systems may not be the sexiest feature, but the underlying technology is so important that it gets developers interested. Apple revealed plans for APFS at its annual Worldwide Developers Conference in June. APFS thoroughly modernizes the way Apple devices track stored information. APFS also adds some really cool features that we haven’t seen before in other file systems.

APFS makes its debut in macOS 10.12 Sierra, currently in development with a release this fall. APFS is optional for developers to use right now, but Apple expects to make it mandatory to use in the future. Since our Mac client is a native app, we like many Apple developers, are boning up now on APFS and what it means.

What is APFS?

Apple hasn’t defined the P in APFS, but that differentiates it from Apple File Service (AFS), a term used to describe older Apple file and network services.

APFS is designed to scale from the smallest Apple device to the biggest. It will work with watchOS, tvOS, iOS and macOS, spanning the entire Apple product line. It’s designed from the get-go to work well on modern Apple device architectures and offers plenty of scalability going forward.

APFS won’t change how you see files. The Finder, the main way you interact with files on your Mac, won’t undergo any major cosmetic changes because of APFS (at least none Apple has told us about yet). Neither will iOS, which largely abstracts file management. This is under-the-hood stuff that tells the computer where to put and how to work with data.

The first implementation of APFS is available to Apple developers in the beta of macOS Sierra 10.12. That’s in the hands of testers now, with a general release planned for this September. Apple isn’t making APFS mandatory today, so don’t worry about this getting in the way if you plan to upgrade to 10.12 once it’s out. Apple plans to deploy APFS across product lines beginning in 2017.

This should be a pretty smooth transition that will make work for developers working on apps, and it’s certainly keeping Apple engineers burning the midnight oil, but we’re not expecting a lot of disruption on your end of things. Though some behaviors are changing in how files are stored on disk which will have an effect on you. So read on for details.

Why did Apple make APFS?

The current file system Apple uses is HFS+. HFS was introduced in 1985, back when the Mac was still new. That’s right, more than thirty years ago now. (HFS+ came later with some improvements for newer Macs.)

To give you an idea of how “state of the art” has changed since then, consider this. My first Mac, which came out late in 1984, had 512 KB of RAM (four times the original Mac’s memory) and a single floppy drive that could store 400K. This computer I’m writing from now has 8 GB of RAM – almost 16 thousand times more RAM than my first Mac – and 512 GB of storage capacity. That’s more than 1.2 million times the capacity of that early machine. Think about that the next time you get a message that your drive is full!

Given the pace of computer technology and development, it’s a bit startling that we still use anything developed so long ago. But that’s how basic and just how important a file system is to a computer’s operation.

HFS+ was cutting-edge for its time, but it’s designed for computers with floppy disk drives and hard drives. Floppies are long gone. The vast majority of Apple devices now use solid state storage like built-in flash and Solid State Drives (SSDs), and those store data differently than hard drives and floppies did.

Why is APFS better?

APFS is designed from the start to better suit the needs of today’s and tomorrow’s devices. It’s thoroughly optimized for devices that use solid state storage – flash and SSDs. These storage technologies work differently than spinning drives do, so it only makes sense to optimize the file system to take advantage.

Apple’s paving the way to store lots more data with APFS. HFS+ supports 32-bit file IDs, for example, while APFS ups that to 64-bit. That means that today, your Mac can keep track of about 4 billion individual pieces of information on its hard drive. APFS ups that to 9 quintillion. That’s a 9 followed by 18 zeroes (actually, much more than that, because of hexadecimal values).

Even though APFS can keep track of orders of magnitude more data than HFS+, you’ll see much faster performance. When you need to save or duplicate files, APFS shares duplicate data between files whenever possible. Instead of duplicating information like HFS+ does, APFS updates metadata links to the actual stored information. Clones, or copies of files or folders can be created instantly. You won’t have to sit and watch as gigabytes of files are duplicated en masse, wasting egregious amounts of space in the process. In fact, clones take up no additional space, since they’re pointing back to the original data! You’ll get much better bang for your storage buck with APFS than HFS+ can manage.

Speaking of space, Space Sharing is another new feature of APFS. This helps the Mac be more effective at managing the free space on its hard drives. You can set up multiple partitions, even multiple file systems, on a single physical device, and all of them can share the same space. You presently have to jump through hoops if you’re resizing partitions and want to re-use de-allocated space. APFS views individual physical devices as “containers,” with multiple “volumes” inside.

Networking is vitally important for almost all computers and computing devices. Over the years there’s been a lot of emphasis on tuning operating system performance for maximum throughput. That’s helpful to developers like us, because we store data in the cloud. But that’s not the whole story. Latency – the amount of time between you telling your computer to do something and when it actually happens – also has a big effect on performance.

Has “the Beachball of Death” ever plagued you? You’ll click a button or try to open something, and the cursor changes to a spinning disk that looks for all the world like a beachball. Apple’s doing a lot more with APFS to make beachballs go away. That’s because they’re prioritizing latency – the amount of time between when you ask your device to do something and when it actually does it.

Apple has found other ways to improve performance wherever possible. Take crash protection, for example. HFS+ uses journaling as a form of crash protection: It keeps track of changes not yet made to the file system in log files. Unfortunately, journaling creates performance overhead. Those log files are constantly being written to and read from. APFS replaces that with a new copy-on-write metadata scheme that’s much more efficient.

Apple is very concerned with user privacy. Their protection of their users’ privacy has occasionally put Apple at loggerheads with governments and individuals who want information off machines. Apple’s taking your privacy seriously with APFS, thanks to much more sophisticated encryption options than before.

Apple’s FileVault technology is its current encryption scheme. FileVault is what’s known as “whole disk” encryption. You turn it on and your Mac encrypts your hard drive. That encrypted data is, for all intents and purposes, unrecognizable unless you enter a password or key to unlock it.

The problem is that FileVault is either on or off, and it’s on or off for the whole volume. So once you’ve unlocked it, your data is potentially vulnerable. APFS still supports whole disk encryption, but it can also encrypt individual files and metadata, with single or multi-key support. That provides additional security for your most sensitive data.

As a backup company, one feature of APFS we’re particularly interested in is its support of snapshots. Snapshots are a pretty standard feature of enterprise backups, but we haven’t seen them yet on the Mac. A snapshot contains pointers to data stored on your disk, providing fast access to data stored on the disk. Because the snapshot contains pointers, not the actual data, it’s compact, and accessing it is very fast.

There’s a lot more under the hood in APFS, but that gives you a broad overview of what it is and why we’re excited. We hope you are too. We’ll be bringing you more details about APFS, and if you have questions, please let us know.

Over the course of the next few months, we‘ll go behind the scenes of our engineering department to show you the impact and opportunity the change to APFS and other new technologies creates for Backblaze. Since our Mac client is a native app, this promises to be pretty cool. Stay tuned.

Peter Cohen
Peter will never give you up, never let you down, never run around or desert you. He also manages the Backblaze blog.

Follow Peter on:
His web site: peter-cohen.com | Twitter: @flargh | LinkedIn: Peter Cohen | Google+: Peter Cohen
Peter Cohen

Latest posts by Peter Cohen (see all)

  • Sean

    “APFS also adds some really cool features that we haven’t seen before in other file systems.” — Yeah, like what?

    64-bit “file IDs” (I assume you mean inodes): ZFS uses 128-bit, so this isn’t exactly revolutionary. Using 128-bit ensures that in *another* 30 years, when storage companies like Backblaze can talk about a zettabyte of data with a straight face and actually contemplate possibly having one on a single logical filesystem, ZFS will be able to support it, long after APFS has gone the way of HFS. Pretty much *all* modern filesystems (ReFS, btrfs, ZFS) are using at least 64-bit inodes.

    Cloning: This is basically just dedup. ZFS and a few other filesystems have supported this for a long time.

    Latency: Wouldn’t this be more related to the I/O scheduler algorithm of the XNU kernel, than the filesystem? It’s not like other filesystems completely ignore latency as a factor, anyway…

    File-level crypto: Supported for years on everything from NTFS to any filesystem on GNU/Linux desktop.

    Snapshots and COW: Another feature pioneered by ZFS that is now seeing wider adoption with btrfs and ReFS.

    Since ZFS was a thing on Mac OS X a long time ago, this isn’t even the first time most of these things are being done *on a Mac*. The only real innovation here is optimizing for SSDs (which I assume is perhaps a little more than just enabling TRIM support), which is only done in a very limited capacity on ZFS, ext4, btrfs, etc. Depending on exactly what SSD optimizations they’re doing, they could genuinely be introducing new filesystem tech. But we’ll have to wait and see. Noteworthy is that OpenZFS on FreeBSD has supported TRIM since 2012.

    In short: almost everything APFS does is not new to the filesystem world at large, and isn’t even really new to the Mac when you consider that ZFS was supported on the Mac starting in 2008-2009 or so.

    If Apple had proceeded full steam with adopting ZFS as their primary filesystem, or even started investing engineering resources into the OpenZFS project to make it better and improve on some of the limitations they identified at the time, then Apple users could have had an APFS-like filesystem much earlier, perhaps as early as 2010.

    What’s old is new again.

  • RustyDreams

    The APIs (Application Interfaces) that apps use to communicate with the file system will not change, with the exception of 4 of them. Almost all apps will have NOTHING to do to run on APFS volumes, their apps will just work. If an app uses one of the 4 APIs not supported, they’ll have a little bit of work to do, but is should not be a major headache to them. Disk Utilities will have to support APFS with all brand new code, and they’ll have to wait until APFS is finished, which it is not, to really get much done on all that new code.

  • Discpad

    Peter Cohen: One huge question I have is, will APFS have a built-in feature to cut down on write amplification on SSD’s? See “The Cleaning Lady and Write Amplification” at:
    http://www.anandtech.com/show/2829/4

    This is a problem when running any version of Windows NT (a/k/a OpenVMS for kiddies), and is why you need a 3rd party file defragger such as Raxco PerfectDisk to manage it. See: “How to Extend the Life of Your SSD Hard Drive” at:
    blog.raxco.com/2012/12/28/how-to-extend-the-life-of-your-ssd-hard-drive-part-1/
    Dan Schwartz,
    Editor, The Hearing Blog
    http://www.TheHearingBlog.com

  • Discpad

    Just a quick correction: HFS+ is not supported on floppy disks. This I know because I still support a Quadra 650 running System 7.6.1 (with AFP/IP from MacOS 8.0) for Pagemaker 7!

  • SteveBlowJobsSucksDickInHell

    So what. Yet another stupid crApple post from backblaze’s own hipster iSheep douche. Why no article about ReFS or Btrfs or ZFS? But as soon as crApple finally gets around to making a modern file system after everyone else already did many years ago, you praise them as if they are doing something innovative. crApple has been behind all other tech companies for almost ten years now. Just stop with all the crApple ass kissing already!

    • Milk Manson

      You’re doing the same exact thing, just pushing in the other direction. Who cares. Get a life.

  • pixelmixture

    does this mean that backblaze boxes will be ported to osx ?

    • Not at this point – we’re quite content managing multiple operating systems for our hardware depending on our needs. APFS certainly paves the way for us to do interesting stuff with the client software in the future, though!

      One of our founders, Brian Wilson, explained a number of years ago his own philosophy towards writing cross platform code. Here’s a link to it if you’d like to read it.

      https://www.backblaze.com/blog/10-rules-for-how-to-write-cross-platform-code/

  • Stefan

    Does this mean that all programs have to be rewritten to work with the new file system?

    • Tex

      No. It’s all behind the scenes. About the only thing that MIGHT have to happen, is the programs need to account for case in paths/file names.

      • Stefan

        OK – thanks.

        • Pretty seamless too, from what I’ve found.