Contents x
How to Use the AWS CLI with Backblaze B2 Cloud Storage
    • Print
    • Dark
      Light
    Contents

    How to Use the AWS CLI with Backblaze B2 Cloud Storage

      • Print
      • Dark
        Light
      Article summary

      Backblaze B2 Cloud Storage has S3 endpoints that you can use in conjunction with the AWS Command Line Interface (CLI) to communicate with and update your Backblaze B2 buckets. After configuration, you can use this tool to transfer data between an S3 bucket and Backblaze B2.

      Enable Backblaze B2

      Before you begin: You must have a Backblaze B2 Cloud Storage account. If you already have a Backblaze account and the left navigation menu contains a B2 Cloud Storage section, your account is already enabled for Backblaze B2.

      1. Sign in to the Backblaze web console.
      2. In the user menu in the upper-right corner of the page, select My Settings.
      3. Under Enabled Products, select the checkbox to enable B2 Cloud Storage.
      4. Review the Terms and Conditions, and click OK to accept them. 

      Create an Application Key

      1. Sign in to the Backblaze web console.

      2. In the left navigation menu under B2 Cloud Storage, click Application Keys.

      3. Click Add a New Application Key, and enter an app key name.
        You cannot search an app key by this name; therefore, app key names are not required to be globally unique. Key names are limited to 100 characters and can contain letters, numbers, and "-", but not I18N characters, such as é, à, and ü.

      4. Select All or select a specific bucket in the Allow Access to Bucket(s) menu.

        If you select a specific bucket, you can also select Allow List All Bucket Names.

        This option is required for the B2 Native API b2_list_buckets and the S3-Compatible API S3 List Buckets operations.

      5. (Optional) Select your access type (Read and Write, Read Only, or Write Only).

      6. (Optional) Enter a file name prefix to restrict application key access only to files with that prefix.
        Depending on what you selected in step #4, this limits application key access to files with the specified prefix for all buckets or just the selected bucket.

      7. (Optional) Enter a positive integer to limit the time, in seconds, before the application key expires.
        The value must be less than 1000 days (in seconds).

      8. Click Create New Key, and note the resulting keyID and applicationKey values.

      Note

      When you create a new app key, the response contains the actual key string, for example N2Zug0evLcHDlh_L0Z0AJhiGGdY. You can always find the keyID on this page, but for security, the applicationKey appears only once. Make sure you copy and securely save this value elsewhere.

      Configure the AWS CLI to Interface With Backblaze B2

      Before you begin: Install the AWS CLI. Use the command aws --version to verify that the CLI is installed.

      1.  Choose whether you want to configure the default AWS CLI profile or a named profile for Backblaze B2.

      2. To configure the default profile, enter the following command:

      aws configure

      3.  To configure a named profile for Backblaze B2, enter the following command:

      aws configure --profile b2

      4. When AWS prompts you for your AWS Access Key ID, enter your BackblazekeyID.

      5. When AWS prompts you for your AWS Secret Access Key, enter your BackblazeapplicationKey.

      Note:
      Your master application key will not work with the Backblaze S3 Compatible API. You must create a new key that is eligible for use. For more information, see this article.


      6. Leave the Default region name and Default output format fields blank.

      7. If you configured a named profile, include --profile b2 when you run AWS CLI commands for Backblaze B2.

      8. If you configured a named profile and receive a NoCredentials error, confirm that your AWS credentials file contains the Backblaze B2 profile credentials.

      # ~/.aws/credentials
[b2]
aws_access_key_id = <Backblaze keyID>
aws_secret_access_key = <Backblaze applicationKey>

      9. If you manually edit your AWS config file, named profiles must use the following format:

      # ~/.aws/config
[profile b2]

      10. Optional: To use --profile b2 without entering --endpoint-url in every command, add the Backblaze B2 S3 endpoint to your AWS config file.

      # ~/.aws/config
[profile b2]
services = b2-s3

[services b2-s3]
s3 = endpoint_url = https://s3.us-west-000.backblazeb2.com

      Replace us-west-000 with the region in your Backblaze B2 S3 Endpoint URL.

      Note:
      Your master application key will not work with the Backblaze S3 Compatible API. You must create a new key that is eligible for use. For more information, see this article.
      Note:
      Do not use aws login to authenticate to Backblaze B2. The aws login command is for AWS Management Console credentials. To use the AWS CLI with Backblaze B2, configure the AWS CLI with your Backblaze application key ID and application key.

      11. Test the integration by entering the following command to list the buckets in your account:

      aws s3 ls --endpoint-url=<S3 Endpoint URL>

      For example:

      aws s3 ls --endpoint-url=https://s3.us-west-000.backblazeb2.com

      If you configured a named profile for Backblaze B2, include --profile b2:

      aws s3 ls --profile b2 --endpoint-url=https://s3.us-west-000.backblazeb2.com

      Example output:

      2023-07-13 13:10:45 S3-SEDemo
2023-07-13 16:23:52 Storagecraft-SETest
2023-07-13 19:50:20 Terraform-SETest
2023-07-13 14:55:06 Venera-SETest

      Use the AWS CLI to Transfer Data to Backblaze B2

      Before you begin: If your data comes from an AWS S3 bucket, add your S3 buckets using the configuration tool.

      1. Copy files to a local directory using the following command:
        aws s3 cp s3://<source_bucket>/ <local_directory> --recursive
      2. Transfer files directly to Backblaze B2 using the following command:
        aws s3 cp <local_directory>/ s3://<destination_bucket>/ --recursive --endpoint-url=<S3 Endpoint URL>

      Create Pre-Signed URLs from the AWS CLI

      You can use the AWS CLI to create a pre-signed URL for sharing an object from a bucket. 

      Enter the following command to generate a pre-signed URL on a file in a bucket that expires in 3,600 seconds (the default value):

      aws s3 presign s3://presignTest/downloadTest.txt --endpoint-url=https://s3.us-west-004.backblazeb2.com/

      A pre-signed URL is returned:

      https://s3.us-west-004.backblazeb2.com/presignTest/downloadTest.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=00445156677e7df0000000007%2F20231018%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20231018T171903Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=78c2d001f4a35a41605ec5b5c3714f47a370be8c5657c22ea3ab36746519afb5

      Pre-Signed URL Example Usage

      Note
      Depending on how you use the pre-signed URL, you may need to enclose it in quotes.
      1. Use a pre-signed URL to get the contents of a file:
        curl "https://s3.us-west-004.backblazeb2.com/presignTest/downloadTest.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=00445156677e7df0000000007%2F20231018%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20231018T171903Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=78c2d001f4a35a41605ec5b5c3714f47a370be8c5657c22ea3ab36746519afb5"
        Example output:
        This is a test!%
      2.  Use a pre-signed URL to save the contents of a file locally (the local file in this example is named “dlTest.txt”):
        curl "https://s3.us-west-004.backblazeb2.com/presignTest/downloadTest.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=00445156677e7df0000000007%2F20231018%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20231018T171903Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=78c2d001f4a35a41605ec5b5c3714f47a370be8c5657c22ea3ab36746519afb5" --output dlTest.txt
      Note
      Uploads are possible using pre-signed URLs but they must be generated with one of our S3 Compatible SDKs.
      Was this article helpful?
      Related articles
      What's Next