Create a Cloud Replication Rule with the Native API

Cloud Replication allows the creation of two rules for each bucket. This means that each file can be used as a source in two rules. Adding any further rules results in an error. Destination buckets can have more than two buckets as the holding bucket for many replicated sources.

There are three steps to use the Backblaze B2 Cloud Storage Cloud Replication feature:

1. Create the Cloud Replication rule.
2. Add a new file or a new version of a file.
3. Check the replication status of the file by looking at the source and destination metadata.

You can also view this video that shows you how to configure Backblaze Cloud Replication using the Native API.

Create a Replication Rule

1. Authenticate using the b2_authorize_account operation.
2. Use the b2_create_bucket operation to create a new bucket in your account and include the replicationConfiguration parameter.
   At least one of the asReplicationSource or asReplicationDestination parameters is required, but they can also both be present.
3. To update your bucket, use the b2_update_bucket operation to update your bucket and include the replicationConfiguration parameter.
4. To list all of the buckets in your account, use the b2_list_bucket operation.

For each replication, the user has to apply one rule for setting up source with asReplicationSource section and second rule for setting up the destination with asReplicationDestination section.

The following example shows a source rule:

{
    "accountId": "12f634bf3cbz",
    "bucketId": "e1256f0973908bfc71ed0c1z",
    "replicationConfiguration": {
        "asReplicationSource": {
            "replicationRules": [
                {
                    "destinationBucketId": "3f46fe8276c62b414506021y",
                    "fileNamePrefix": "",
                    "includeExistingFiles": false,
                    "isEnabled": true,
                    "priority": 1,
                    "replicationRuleName": "replication-us-east"
                }
            ],
            "sourceApplicationKeyId": "00512f95cf4dcf0000000004z"
        },
    }
}

The following example shows a destination configuration that is deployed on the destination account. The source of this bucket is set up by source bucket owner in source configuration, so the destination needs only the source key ID to map the two buckets.

{
	"accountId": "12f634bf3cbz",
	"bucketId": "e1256f0973908bfc71ed0c1z",
	"replicationConfiguration": {
		"asReplicationDestination": {
			"sourceToDestinationKeyMapping": {
				"00512f95cf4dcf0000000004z": "00512f95cf4dcf0000000004y",
			}
		}
	}
}

The source replication configuration is set within the asReplicationSource section, and the destination replication app key mapping is set within the asReplicationDestination section.

Cloud Replication rules created by the information above can then be turned off by calling b2_update_bucket setting the isEnabled parameter to false. To delete a rule, call b2_list_buckets, if necessary, to obtain the bucket's list of replication rules, remove the rule from the replicationRules JSON array, and call b2_update_bucket, supplying the modified replicationRules. Note, the API layer allows the rules to be turned off or deleted even when created from the Backblaze web UI.

Call b2_list_buckets to retrieve all of the replication rules associated with one or more buckets. Please keep in mind that deleted Replication Rules are not returned.

Each replication rule has the following fields:

• destinationBucketId: The ID of a bucket to replicate new file uploads to (cannot be the same as the source bucket).
• fileNamePrefix: Only files matching this prefix will be replicated.
• priority: Priority for resolving conflicts if two or more rules conflict. In general a file will be replicated according to all rules. However, if two or more rules have the same destination, then the priority value will be used to select which rule to consider. A higher priority value wins.
• isEnabled: true/false boolean indicating if the rule is enabled. If the rule is disabled it will not be applied. In the front end, this value represents the unpaused state.
• includeExistingFiles: true/false boolean indicating if existing files in the bucket will be replicated.
• replicationRuleName: Customer provided string that serves as a name or description for the rule.

Cloud Replication and Authentication Keys

When creating a replication rule, you must provide an Authentication Key that has read-access for the source bucket and write-access for the destination bucket. Also, keep in mind that failures may occur due to billing issues. Make sure the source and destination are set up to accommodate the fees associated with Cloud Replication. Contact support if you run into any issues.

To successfully create a working replication rule, the authentication keys must have specific rights.

For Source buckets, the Authentication Key specified must have the following rights:

• readFiles
• readFileLegalHolds
• readFileRetentions

For Destination buckets, the Authentication Key specified must have the following rights:

• writeFiles
• writeFileLegalHolds
• writeFileRetentions

File Name Prefixes

The fileNamePrefix parameter allows API calls to specify a prefix to restrict replication to files with matching names, this feature is not available in the Backblaze web UI.

Add a New File or File Version

There is no difference to the process of uploading files to bucket(s) with Cloud Replication rules. But keep the following items in mind:

• New files, and new versions of existing files, will be replicated regardless of whether they are created via the API or the Backblaze web UI.
• Keep in mind that the replication engine is on a distributed system, and so the time to complete replication is based on the number of other replication jobs scheduled, number of files to replicate, and the size of the files to replicate.

Check Replication Status

To see the replication status and if the file is a replica itself, call b2_get_file_info. The field replicationStatus provides the given file's replication status. Each file supports two rules that can be defined.

The replicationStatus may have one of the following values:

• pending: This file is in process of being replicated. If there are two rules, at least one of the rules is in process. Check again later to see if it has left this status.
• completed: This represents a successful replication. If two rules are configured, both rules have completed successfully.
• failed: A non-recoverable error has occurred, such as insufficient permissions. The system will not try again to process this file. If two rules are configured, at least one has failed.
• replica: This file was created by the replication process. Note that any replica files cannot be used as the source for further replication.

Additional Notes

• When creating the replication rules the writeBucketReplication must be set on the Authentication Key (readBucketReplications provides the ability to view but not change replication rules.)
• To replicate all files in a bucket, compared to only new files being replicated, set the value of includeExistingFiles to true. See below for more details to the fields that can be set for cloud replication rules.
• Once a file has begun replicating; transitioned to a pending replication status, the replication rule's Source Application Key at the time of transition of will be used to perform the replication.
• If the replication rule is deleted or updated after the file has transitioned into a pending replication status, the replication will still be performed. If failing the replication is desired, deleting the Source Application Key, or the Destination sourceToDestinationKeyMapping will fail the replication.