Block-level Deduplication, Compression, and Encrypted Backup With Duplicati

When you’re responsible for backing up your organization’s servers, workstations, and NAS storage systems, choosing the best mix of protection vs. solution cost can feel like a high-wire act. One way to meet this challenge is considering how your backups are stored: In contrast to file-based, or storage sync backup solutions, traditional block-level backup paired with deduplication and compression will keep your data footprint as small and efficient as possible—enhancing your ability to back up everything you might want to.

With its block-level approach, Duplicati is one tool that, depending on your use case, can help you avoid picking and choosing which systems to protect. This post explores what Duplicati does, and how it might work for you.

Duplicati From 20,000 Feet

Duplicati is an open-source project that many IT administrators have found to meet the same specification of commercial backup solutions—including an excellent interface to manage and monitor underlying backup functions, and encryption so your backups can only be restored and accessed by you—without prohibitive complexity and pricing.

There are no support or maintenance contracts required, and no charge to use the software so you can install it on every system needing protection. When paired with a cloud storage solution, Duplicati can quickly and affordably deliver backup protection for every system you are responsible for.

Duplicati is a free, open-source backup client for macOS, Windows, and Linux that securely stores encrypted, incremental, compressed backups in cloud storage like Backblaze B2 Cloud Storage.

Duplicati Features at a Glance

Here’s a brief review of Duplicati’s key features to help in your evaluation. If you’re already sold on its functionality, skip ahead to the next section.

Installation and Architecture

As an open-source tool, Duplicati is free to download and use, does not require a maintenance contract, and yet is frequently updated with installers for macOS, Windows, and Linux systems, as well as for NAS storage systems such as Synology. Once installed, you access Duplicati’s features through a web interface, mobile devices, or via command-line, if you prefer. It’s a good practice to install the same tool on all of your servers, workstations, and NAS systems to make administration as easy and uniform as possible.

Backup Formats

Duplicati gathers all files to be backed up, deduplicates and compresses them, then sends them to your backup location in blocks or chunks to be stored for maximum efficiency. The first time you run a Duplicati backup job it will perform a full backup of your system, then each backup job after that will send incremental backups of the changed files, further sparing your bandwidth.

Security and Encryption

Backups stored using Duplicati can be encrypted with AES-256 encryption with custom pass-phrases, or integrate with your GPG security toolchain. This ensures that only your team can decrypt the backups and restore the files, preserving data security.

Other Advantages

While ease of use, storage format, and security will likely apply to just about every use case, Duplicati offers a number of other features that can lighten your administrative load and offer peace of mind. For instance, Duplicati lets you establish schedules to automate your backup jobs and how you’d like to be notified on job completion or alerted if there’s a problem. If you have a distributed, or highly mobile team, this can be hugely helpful. Additionally, Duplicati can also be set to periodically download a random set of backup files, restore them, and verify their integrity again. Any backup strategy should include a testing regimen, and Duplicati provides an easy way to set one up in your workflow.

Installing and Configuring Duplicati

This example outlines the installation and configuration of Duplicati using cloud storage for backups—in this case, Backblaze B2.

Begin by creating a dedicated storage bucket, and ensuring that it’s set to private.

Next, navigate to Backblaze B2, then App Keys, then Add a New Application Key. Once there, generate an Application Key and make sure that it has read and write access to the dedicated storage bucket you’ve made:

Before exiting, make a note of the bucket name, Application Key, and ID for your configuration work in Duplicati.

Configuring Your First Duplicati Backup

Duplicati’s manual will help guide you through installation, which begins with downloading the most recent build for your platform and following the wizard-style installation steps.

Once installed, Duplicati will ask you to set a user password, then present you with the main screen, ready to configure your first backup with Backblaze B2. Select Add Backup, then Add a New Backup, then Configure a New Backup.

Duplicati is organized around the principle of “tasks” that specify a backup job for a single system. When setting up a task, you’ll be asked to name the backup job, to name the folder that will hold your backups, to specify optional encryption settings and a passphrase, and then specify the target destination, in this case Backblaze B2.

Enter the credentials and bucket information you created earlier.

Be sure to test the connection before going further.

Click next to select the systems and folders to include in this backup task, then next again to fine tune how often and when you want this backup job to run.

Finally, save your backup job.

Now, back on the Duplicati home screen, you can see your defined backup job and inspect its progress:

As you can see when you navigate back to Backblaze, the actual backup blocks are compressed files that can only be restored and decrypted by you.

In only a few minutes, you’ve configured your first Duplicati backup task and are ready to start protecting the rest of your server, workstation, and NAS storage fleet.

Some Notes on Backing Up to Cloud Storage

As you’re considering where to store your backups—or as you’re trying to convince others of the value of off-site cloud storage—there are two things to keep in mind: First, using cloud storage will put you on the right side of your budget. Cloud storage allows you to keep your storage in operating expense, which means your budget can be scaled up and down as needed, and you don’t have money needlessly tied up in aging tech infrastructure. (You can read more about a CapEx vs. OpEx strategy, here.) Second, cloud storage helps you ensure that your backups are off-site to avoid any local system outage, and yet they also remain immediately accessible so that your recovery can begin immediately. If you liked the ease of working with Backblaze B2 in this example, you can learn more about the service, here.

Does Duplicati Meet Your Checklist?

Duplicati’s block-level backup, with deduplication, compression, and encryption could be exactly what your organization needs, and eliminating license, support, and maintenance costs are highly attractive as well. When paired with cloud storage like Backblaze B2, which also supports server-side encryption using AES-256, you can affordably protect every system in your organization.

Do these features meet your checklist? How was your experience getting it set up? Let us know in the comments.


About Skip Levens

Skip drives Backblaze's outreach to Creative and Media and Entertainment customers - and has been a long-time champion of intelligent digital storytelling and television and film production workflows.