{"id":105848,"date":"2022-05-26T08:53:55","date_gmt":"2022-05-26T15:53:55","guid":{"rendered":"https:\/\/www.backblaze.com\/blog\/?p=105848"},"modified":"2022-12-16T13:14:32","modified_gmt":"2022-12-16T21:14:32","slug":"looking-forward-to-backblaze-cloud-replication-everything-you-need-to-know","status":"publish","type":"post","link":"https:\/\/www.backblaze.com\/blog\/looking-forward-to-backblaze-cloud-replication-everything-you-need-to-know\/","title":{"rendered":"Looking Forward to Backblaze Cloud Replication: Everything You Need to Know"},"content":{"rendered":"

\"\"<\/p>\n

Editor\u2019s note, June 7, 2022:<\/strong> Backblaze Cloud Replication is now generally available. Read the announcement to learn more.<\/a><\/div>\n

Backblaze Cloud Replication<\/a>\u2014currently in private beta\u2014enables Backblaze customers to store files in multiple regions, or create multiple copies of files in one region, across the Backblaze Storage Cloud. This capability, as we explained in an earlier blog post<\/a>, allows you to create geographically separate copies of data for compliance and continuity, keep data closer to its consumers, or maintain a live copy of production data for testing and staging. Today we\u2019ll look at how<\/em> you can get started with Cloud Replication, so you\u2019ll be ready for its release, likely early next month.<\/p>\n

Backblaze Cloud Replication: The Basics<\/h2>\n

Backblaze B2 Cloud Storage organizes data into files<\/em> (equivalent to Amazon S3\u2019s objects<\/em>) in buckets<\/em>. Very simply, Cloud Replication allows you to create rules<\/em> that control replication of files from a source<\/em> bucket to a destination<\/em> bucket. The source and destination buckets can be in the same or different accounts, or in the same or different regions.<\/p>\n

Here\u2019s a simple example: Suppose I want to replicate files from my-production-bucket<\/code> to my-staging-bucket<\/code> in the same account, so I can run acceptance tests on an application with real-life data. Using either the Backblaze web interface or the B2 Native API, I would simply create a Cloud Replication rule specifying the source and destination buckets in my account. Let\u2019s walk through a couple of examples in each interface.<\/p>\n

Cloud Replication via the Web Interface<\/h3>\n

Log in to the account containing the source bucket for your replication rule. Note that the account must have a payment method configured to participate in replication. Cloud Replication will be accessible via a new item in the B2 Cloud Storage<\/strong> menu on the left of the web interface:<\/p>\n

\"\"<\/p>\n

Clicking Cloud Replication<\/strong> opens a new page in the web interface:<\/p>\n

\"\"<\/p>\n

Click Replicate Your Data<\/strong> to create a new replication rule:<\/p>\n

\"\"<\/p>\n

Configuring Replication Within the Same Account<\/h3>\n

To implement the simple rule, \u201creplicate files from my-production-bucket<\/code> to my-staging-bucket<\/code> in the same account,\u201d all you need to do is select the source bucket, set the destination region the same as the source region, and select or create the destination bucket:<\/p>\n

\"\"<\/p>\n

Configuring Replication to a Different Account<\/h3>\n

To replicate data via the web interface to a different account, you must be able to log in to the destination account. Click Authenticate an existing account<\/strong> to log in. Note that the destination account must be enabled for Backblaze B2 and, again, must have a payment method configured:<\/p>\n

\"\"<\/p>\n

After authenticating, you must select a bucket in the destination account. The process is the same whether the destination account is in the same or a different region:<\/p>\n

\"\"<\/p>\n

Note that, currently, you may configure a bucket as a source in a maximum of two replication rules. A bucket can be configured as a destination in any number of rules.<\/p>\n

Once you\u2019ve created the rule, it is accessible via the web interface. You can pause<\/strong> a running rule, run<\/strong> a paused rule, or delete<\/strong> the rule altogether:<\/p>\n

\"\"<\/p>\n

Replicating Data<\/h3>\n

Once you have created the replication rule, you can manipulate files in the source bucket as you normally would. By default, existing files in the source bucket will be copied to the destination bucket. New files, and new versions of existing files, in the source bucket will be replicated regardless of whether they are created via the Backblaze S3 Compatible API, the B2 Native API, or the Backblaze web interface. Note that the replication engine runs on a distributed system, so the time to complete replication is based on the number of other replication jobs scheduled, the number of files to replicate, and the size of the files to replicate.<\/p>\n

Checking Replication Status<\/h3>\n

Click on a source or destination file in the web interface to see its details page. The file\u2019s replication status is at the bottom of the list of attributes:<\/p>\n

\"\"<\/p>\n

There are four possible values of replication status:<\/p>\n

    \n
  • pending<\/code>: The file is in the process of being replicated. If there are two rules, at least one of the rules is processing. (Reminder: Currently, you may configure a bucket as a source in a maximum of two replication rules.) Check again later to see if it has left this status.<\/li>\n
  • completed<\/code>: This status represents a successful replication. If two rules are configured, both rules have completed successfully.<\/li>\n
  • failed<\/code>: A non-recoverable error has occurred, such as insufficient permissions to write the file into the destination bucket. The system will not try again to process this file. If two rules are configured, at least one has failed.<\/li>\n
  • replica<\/code>: This file was created by the replication process. Note that replica files cannot be used as the source for further replication.<\/li>\n<\/ul>\n

    Cloud Replication and Application Keys<\/h3>\n

    There\u2019s one more detail to examine in the web interface before we move on to the API. Creating a replication rule creates up to two Application Keys; one with read permissions for the source bucket, if the source bucket is not already associated with an Application Key, and one with write permissions for the destination bucket.<\/p>\n

    The keys are visible in the App Keys<\/strong> page of the web interface:<\/p>\n

    \"\"<\/p>\n

    You don\u2019t need to worry about these keys if you are using the web interface, but it is useful to see how the pieces fit together if you are planning to go on to use the B2 Native API to configure Cloud Replication.<\/p>\n

    This short video walks you through setting up Cloud Replication in the web interface:<\/p>\n