Backblaze B2 Quick Start: Using Python With the Backblaze S3 Compatible API

This tutorial provides developers familiar with the Python language with prebuilt code that accesses hosted sample data. In just a few minutes, you’ll get hands-on experience working with Backblaze B2 Cloud Storage. The Backblaze B2 Quick Start includes three options for developers to become proficient with the prebuilt code. First, the open-source code is on GitHub, including helpful commenting in-line. Second, written instructions are provided below for a guided walk-through. Third, video-guided walk-throughs are also embedded below.

KEY TOPICS

1. Overview: Exercise Resources Explained

2. Sample Application: GitHub, Sample Code Structure, Detailed Walk-Throughs of Multiple API Functions

1. OVERVIEW

Hello, and welcome to this Backblaze B2 Quick Start using the Python programming language with the Backblaze S3 Compatible API.

The intended audience are developers who have experience with the Python programming language.

The following instructions provide you with details on using prebuilt Python code that we have shared on GitHub for you to download.

You can get started free of charge by creating a Backblaze B2 account with 10GB of storage at no cost (and downloading up to 1GB/day for free). The instructions below first walk through the prebuilt code on GitHub, making API calls against a prebuilt media application in a pre-staged Backblaze B2 bucket. The API calls against the pre-staged Backblaze B2 bucket are read-only and we have shared the necessary read-only keys in the GitHub project.

Later instructions provide you a guided walk-through of the prebuilt code on GitHub that includes API calls that create buckets, upload files, and delete files and buckets. For this second half of the prebuilt code, you will need to create your own Backblaze account and buckets, which you can do free of charge.

This is part of a comprehensive set of resources that are available to you.

RESOURCES:

  1. Sample Application: open-source and shared on GitHub for you to download: https://github.com/backblaze-b2-samples/b2-python-s3-sample
  2. Video Code Walk-throughs of Sample Application: For you to share and rewatch on demand. See videos embedded inline below.
  3. Hosted Sample Data: A media application with application keys shared for read-only access
  4. Guided Instructions: These pages that you are currently reading. These instructions guide you on how to download the sample code, run it yourself, and then use the code as you see fit, including incorporating it into your own applications.

These resources initially use our keys, buckets, and sample data. However, 100% of the prebuilt code on GitHub is immediately usable with keys, buckets, and data in your own Backblaze B2 accounts!

Please share these resources with your friends and coworkers.

As you build on B2 Cloud Storage, we'd very much love to hear what you do with it. If you would like to contribute to this project on GitHub, we welcome your participation via pull requests.

Video 1: Connecting to Backblaze B2

This first video provides an alternative way of exploring the sample application and reviewing the topics that follow immediately below:

2. SAMPLE APPLICATION

GitHub

As highlighted above, source code used in these exercises is open-sourced and shared on GitHub for you to download. Download here: https://github.com/backblaze-b2-samples/b2-python-s3-sample

Sample Code Structure (Including .env File)

This sample application code can be downloaded and executed as is.

Among the GitHub files are two critical files:

Typically, GitHub repositories do not contain .env files, but these credentials are limited to read-only access of public data, so they are safe to share.

If you are altogether new to developing with Python, the Python IDE used in our video walk-through of the code uses PyCharm Community Edition IDE. PyCharm is not open-source, however the Community Edition is available free for your use and can be downloaded from here:
https://www.jetbrains.com/pycharm/download/

Hosted Media Application

The Backblaze B2 bucket that we will be using is configured with PUBLIC access. This means that all of the objects in the Backblaze B2 sample data bucket can be downloaded via regular, unsigned HTTP requests. HTTP requests are the basis of the internet and are used by web browsers to request files that browsers display.

The bucket includes a browser photo viewer application that you can view here:
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/photos.html

Note, this B2 Cloud Storage bucket contains five .jpg photo files visible at the link above, plus all of the associated HTML, CSS, and JavaScript files for the media viewer application. The JavaScript implementation uses the open-source Swiper project. You can find the Swiper source at: https://github.com/nolimits4web/swiper.

You’ll find it useful to open sample.py in a code editor so you can refer to it as you read the following explanation.

Function get_b2_resource()

In the sample application, the critical logic showing how to get a connection to B2 Cloud Storage is in the function named get_b2_resource(). The following are the lines of code you will find in the sample application defining the function get_b2_resource():

# Return a boto3 resource object for B2 service
def get_b2_resource(endpoint, key_id, application_key):
    b2 = boto3.resource(service_name='s3',
                        endpoint_url=endpoint,     # Backblaze endpoint
                        aws_access_key_id=key_id,  # Backblaze keyID
                        aws_secret_access_key=application_key,  
                                                   # Backblaze applicationKey
                        config=Config(
                            signature_version='s3v4'))
    return b2

As you can see in the code, the sample application uses the open-source boto3 library. On execution the function returns a boto3 service resource. In the code the value returned is stored in a variable named b2. Referencing this b2 variable you can then call on the B2 Cloud Storage service actions, sub-resources and collections. For a full set of supported calls supported by the B2 Cloud Storage service see the documentation here.

The sample application includes the following among required import statements at the top of the application:

import boto3

For details on downloading and installing the current version of boto3, please see: https://pypi.org/project/boto3/

Variations of this logic can be deployed wherever you run your code. This might be on local machines, running in browsers, or running on servers. Regardless you're going to be storing and interacting with files that are stored out in the cloud on the Backblaze B2 service.

The get_b2_resource() function passes five parameters to boto3.resource():

The first named parameter, service_name, selected boto3’s ‘s3’ service.

Note: Although the value being passed is the literal string 's3', all API calls will go to the Backblaze B2 service via the Backblaze S3 Compatible API.

API request routing and authentication are governed by the remaining four parameters and settings in the B2 Cloud Storage bucket:

The middle two parameters both start with the prefix aws_, however, the values being passed apply to the Backblaze B2 service.

Overriding the default Amazon S3 endpoint URL allows us to use boto3 to connect to Backblaze B2.

In the sample code, the values being passed via 3 of these parameters are retrieved from the .env file.

The names of all parameters are hardcoded in the boto3 library. The middle two both start with the prefix aws_ however the values being passed are from the Backblaze B2 service.

CONSTANTS AND .env FILE

The following are the the lines of code you will find in the sample application file .env():

# B2 API endpoint for buckets with sample data
ENDPOINT='https://s3.us-west-002.backblazeb2.com'

# Following 2 application key pairs provide read-only access
# to buckets with sample data

# Bucket with sample data and PUBLIC access
KEY_ID_RO='0027464dd94917b0000000001'
APPLICATION_KEY_RO='K002WU+TkHXkksxIqI6IDa/X7dsN9Cw'

# Bucket with sample data and PRIVATE access
KEY_ID_PRIVATE_RO='0027464dd94917b0000000002'
APPLICATION_KEY_PRIVATE_RO='K002ckrkS/KpaRA9IFzC3xyIn79ALw4'

# Variables below for functions that require write access!
# You must set these values using your own Backblaze account
# 1. Retrieve B2 API Endpoint for region containing your Bucket
# 2. Create Key Pair in Backblaze Console
#    Direct Link here to "App Keys" page https://secure.backblaze.com/app_keys.htm
#    In Backblaze console, select "App Keys" on left-side nav (3rd up from bottom)
#    Then select "Add a New Application Key" then "Read and Write" re "Type of Access"
#    In Backblaze console, values are labeled as keyID and applicationKey respectively

ENDPOINT_URL_YOUR_BUCKET='<ENTER YOUR B2 API ENDPOINT HERE!>'
KEY_ID_YOUR_ACCOUNT='<ENTER YOUR keyID HERE!>'
APPLICATION_KEY_YOUR_ACCOUNT='<ENTER YOUR applicationKey HERE!>'

These constants are stored separately in the .env file because it is best practice to never check key values and other secrets into a repository. We are making an exception in this case in order for you to have a very fast development startup experience. In your own production applications you should add .env to .gitignore to avoid checking in the .env file. Please note that in .env the names of both constants for the key values end with an _ro for “read only” to indicate that these keys are related to read-only access. Please also note the comments in the .env file that identify how the key variables are labeled in the Backblaze console: keyID and applicationKey. When you use this code against buckets in your own account you will generate your own keys via the “App Keys” section under “Account” in the Backblaze console.

When get_b2_resource() is called it invokes the boto3.resource() method which returns a reference to the Backblaze B2 ServiceResource object. With the Backblaze B2 ServiceResource object, your code can now call additional actions and sub-resources defined on it in boto3. For details on actions and sub-resources see the Boto3 Docs. For the list of S3 calls supported by B2 Cloud Storage, see the docs for the Backblaze S3 Compatible API.

This script is designed to successfully execute without any parameters. In this sample application code, the main() function calls get_b2_resource(), then, by default, calls list_object_keys().

When executed, the script prints the names, or “keys,” of the objects in the specified bucket. By default the bucket referenced is set on the constant named PUBLIC_BUCKET_NAME. See “Hosted media application” above for details of the sample bucket.

Among the keys displayed, you will see the following five photos:

Video 2: Reviewing Two functions: Sample Application main() and list_object_keys()

This next video provides an alternative way of exploring the sample application and reviewing the topics that follow immediately below:

Function main()

The following are a sample of the lines of code you will find in the sample application defining the function main():

"""
Python main() 

Basic execution setup
Then conditional blocks executing based on command-line arguments passed as input.
"""
def main():
    args = sys.argv[1:]  # retrieve command-line arguments passed to the script

    load_dotenv()                   # load environment variables from file .env

    # get environment variables from file .env
    endpoint = os.getenv("ENDPOINT")  # Backblaze endpoint
    key_id_ro = os.getenv("KEY_ID_RO")  # Backblaze keyID
    application_key_ro = os.getenv("APPLICATION_KEY_RO") # Backblaze applicationKey

    # Call function to return reference to B2 service
    b2 = get_b2_resource(endpoint, key_id_ro, application_key_ro)

    client = boto3.client(service_name='s3',
                          endpoint_url=endpoint,
                          aws_access_key_id=key_id_ro,
                          aws_secret_access_key=application_key_ro)
    # pyboto3 provides Pythonic Interface for typehint for autocomplete in pycharm
    """ :type : pyboto3.s3 """

    # get environment variables from file .env
    # Backblaze keyID
    key_id_private_ro = os.getenv("KEY_ID_PRIVATE_RO")
    # Backblaze applicationKey
    application_key_private_ro = os.getenv("APPLICATION_KEY_PRIVATE_RO")

    # Call function to return reference to B2 service using a second set of keys
    b2_private = get_b2_resource(endpoint, 
                                 key_id_private_ro, application_key_private_ro)

    # 01 - list_object_keys
    if len(args) == 0 or (len(args) == 1 and args[0] == '01'):
        # Call function to return list of object 'keys'
        bucket_object_keys = list_object_keys(PUBLIC_BUCKET_NAME, b2)
        for key in bucket_object_keys:
            print(key)

        print('\nBUCKET ', PUBLIC_BUCKET_NAME, ' CONTAINS ', 
              len(bucket_object_keys), ' OBJECTS')

    # 02 - List Objects formatted as browsable url
    # IF *PUBLIC* BUCKET, PRINT OUTPUTS BROWSABLE URL FOR EACH FILE IN THE BUCKET
    elif len(args) == 1 and (args[0] == '02' or args[0] == '02PUB'):
        # Call function to return list of object 'keys' concatenated into 
        # friendly urls
        browsable_urls = list_objects_browsable_url(PUBLIC_BUCKET_NAME, endpoint, b2)
        for key in browsable_urls:
            print(key)

        print('\nBUCKET ', PUBLIC_BUCKET_NAME, ' CONTAINS ', 
              len(browsable_urls), ' OBJECTS')

In Python programs, the main() function is both the top-level environment and the starting point of logic each time the program is executed. When the program is run, the Python interpreter runs the code sequentially.

The first statement in main() reflects that the sample application is architected to optionally take parameters as input. The following line of code retrieves the command-line arguments passed to the script and stores them in the variable args. In later lines of code there are several conditional blocks testing for various expected command-line arguments expected to be passed as input.

    args = sys.argv[1:]  # retrieve command-line arguments passed to the script

The sample application includes the following among required import statements at the top of the application:

import sys

The next lines of code load the values from the .env file. How this application is using the .env file was discussed earlier under the heading "CONSTANTS AND .env FILE" above. The last three lines below load each of the string values stored in the .env file into local variables:

    load_dotenv()   # load environment variables from file .env

    # get environment variables from file .env
    endpoint = os.getenv("ENDPOINT")  # Backblaze endpoint
    key_id_ro = os.getenv("KEY_ID_RO")  # Backblaze keyID
    application_key_ro = os.getenv("APPLICATION_KEY_RO") # Backblaze applicationKey

The sample application includes the following among required import statements at the top of the application:

import boto3  # REQUIRED! - Details here: https://pypi.org/project/boto3/
from botocore.exceptions import ClientError
from botocore.config import Config
from dotenv import load_dotenv  # Project Must install Python Package:  python-dotenv
import os
import sys

The next line of code calls the function get_b2_resource(). The logic inside this function was discussed earlier under the heading "Function get_b2_resource()" above. Note that in invoking this function, this line of code passes as input the values retrieved from the .env file. This function returns a resource object for the B2 Cloud Storage service. The reference to this resource object is stored in variable b2 which will be referenced in subsequent lines of code here in the sample application.

    # Call function to return reference to B2 service
    b2 = get_b2_resource(endpoint, key_id_ro, application_key_ro)

There are additional setup statements at the top of main(). They will be discussed later when we review the code that use them.

The remainder of the logic in the main() function is in conditional blocks. With each execution of the sample program, only one conditional block will execute. In turn, each conditional block executes a call to the B2 Cloud Storage service.

Since each conditional block executes a sample application function, the remainder of the logic in the main() function will be reviewed below in conjunction with the description of the functions that they call.

Function list_object_keys()

First up is a read-only operation to list_object_keys().

For the sample application, the default case when no arguments are passed is the execution of the first conditional block in main() function. Following is the full logic of this first block.

    # 01 - list_object_keys
    if len(args) == 1 and args[0] == '01':
        # Call function to return list of object 'keys'
        bucket_object_keys = list_object_keys(PUBLIC_BUCKET_NAME, b2)
        for key in bucket_object_keys:
            print(key)

        print('\nBUCKET ', PUBLIC_BUCKET_NAME, ' CONTAINS ', 
              len(bucket_object_keys), ' OBJECTS')

Following is the conditional logic for the first block. This is the default case if no arguments are passed (len(args) == 0), or if input parameter of "01" is passed (if (len(args) == 1 and args[0] == '01')). When logic for this block resolves to true, it will execute the sample application function named list_object_keys().

      # 01 - list_object_keys
    if len(args) == 1 and args[0] == '01':

When this block executes, it in turn executes the following call on the function list_object_keys(). Note that this call is passing in two parameters: PUBLIC_BUCKET_NAME and b2. The input parameter b2 and how it was created was described above.

    # Call function to return list of object 'keys'
    bucket_object_keys = list_object_keys(PUBLIC_BUCKET_NAME, b2)

PUBLIC_BUCKET_NAME is a constant defined at the top of the sample application as follows:

    # Bucket with Sample Data **PUBLIC**
    PUBLIC_BUCKET_NAME = 'developer-b2-quick-start2'

Before discussing remaining logic in this conditional block in main(), let's first review the logic inside function list_object_keys().

# List the keys of the objects in the specified bucket 
def list_object_keys(bucket, b2):
    try:
        response = b2.Bucket(bucket).objects.all()

        return_list = []               # create empty list
        for object in response:        # iterate over response
            return_list.append(object.key) # for each item in response,
                                       # append object.key to list   
        return return_list             # return list of keys from response 

    except ClientError as ce:
        print('error', ce)

The following is sample output from execution of list_object_keys() (after print() output in main()):

"C:\temp\B2_Python_Quick_Start\python.exe" C:/SAMPLES/B2_Python_Quick_Start/sample.py 
album/.bzEmpty
album/assets/.bzEmpty
album/assets/carousel-slider.uiinitiative.com-index.ed866659.css
album/assets/index.b1995cd6.js
album/assets/swiper@8.0.3~swiper-bundle.min.css
album/assets/swiper@8.0.3~swiper-bundle.min.js
album/assets/vendor.50b6404e.js
album/carousel.html
album/photos.html
beach.jpg
bobcat.jpg
coconuts.jpg
lake.jpg
sunset.jpg

BUCKET developer-b2-quick-start CONTAINS 14 OBJECTS

Process finished with exit code 0

So, let's step through the logic of the function list_object_keys(). First, the function's signature specifies two input arguments, bucket and b2. As we saw in the calling code back in main(), the value passed to the bucket argument will be a string containing the name of the bucket to be referenced. And the b2 argument must be a reference to a valid Backblaze B2 resource service object.

# List the keys of the objects in the specified bucket 
def list_object_keys(bucket, b2):

Inside the function there are two blocks, try: and except. For our review of the functions in this sample application, we will focus solely on the logic in the try: blocks. For the sample application, the except blocks contain only simple print() statements. For details on error handling in boto3, please see the documentation here.

# List the keys of the objects in the specified bucket 
def list_object_keys(bucket, b2):
    try:
        ...

    except ClientError as ce:
        print('error', ce)

The sample application includes the following among required statements at the top of the application:
from botocore.exceptions import ClientError.

Now let's step through the logic in the try block. First, we take the two input arguments and using method-chaining syntax, return back an iterable collection of ObjectSummary resources and store them in a local variable named response. Reading this first line from left to right, we take the b2 input argument and call on it the constructor for the bucket sub-resource, passing in as input the bucket input argument. With that the chain now has reference to a B2 Cloud Storage bucket resource object. Continuing the chain, the logic next references the bucket resource's available objects collection and calls on it the function all(). In processing this chain, the logic communicates with the backend Backblaze B2 service and the Backblaze B2 service sends back the iterable collection of ObjectSummary resources which our logic now stores locally in variable response.

        response = b2.Bucket(bucket).objects.all()

The intent of the list_object_keys() function is to return back to the caller an iterable collection of object key values. The next three lines start by declaring a local variable return_list as an empty list which the next two lines will populate as the logic iterates over the response collection. The next line uses a for statement to iterate over the response, extracting each object. The third line uses the append() method on the list return_list to add the key value on each object to the collection.

    return_list = []               # create empty list
    for object in response:        # iterate over response
        return_list.append(object.key) # for each item in response,
                                   # append object.key to list 

The function's processing is now complete and the last line uses the return statement to return back the reference to return_list.

    return return_list             # return list of objects from response

For the sample application, processing now continues back in the main() function conditional block that called list_object_keys(PUBLIC_BUCKET_NAME, b2). The following is the full logic of this calling block.

    # 01 - list_object_keys
    if len(args) == 1 and args[0] == '01':
        # Call function to return list of object 'keys'
        bucket_object_keys = list_object_keys(PUBLIC_BUCKET_NAME, b2)
        for key in bucket_object_keys:
            print(key)

        print('\nBUCKET ', PUBLIC_BUCKET_NAME, ' CONTAINS ', 
              len(bucket_object_keys), ' OBJECTS')

Thus, the execution of this line of code is now complete. And the variable bucket_object_keys now contains an iterable collection of object key values.

    bucket_object_keys = list_object_keys(PUBLIC_BUCKET_NAME, b2)

The remainder of the logic here in the conditional block generates print() output to display the returned results.

The next two lines use a for statement to first iterate over the bucket_object_keys extracting from each a reference to each object in the collection. The second line uses a print() statement to display the key of each object in the collection.

    for key in bucket_object_keys:
        print(key)

Lastly, the conditional block closes with the following print() statement to display the PUBLIC_BUCKET_NAME and a count of the number of keys returned by the call using logic of len(bucket_object_keys). These two variable values are concatenated with 3 literal strings to display output such as:
BUCKET developer-b2-quick-start CONTAINS 14 OBJECTS

    print('\nBUCKET ', PUBLIC_BUCKET_NAME, ' CONTAINS ', 
          len(bucket_object_keys), ' OBJECTS')

Recapping and repeating, the following is sample output from the execution of list_object_keys() and the print() statements following its execution in main()):

python sample.py 
album/.bzEmpty
album/assets/.bzEmpty
album/assets/carousel-slider.uiinitiative.com-index.ed866659.css
album/assets/index.b1995cd6.js
album/assets/swiper@8.0.3~swiper-bundle.min.css
album/assets/swiper@8.0.3~swiper-bundle.min.js
album/assets/vendor.50b6404e.js
album/carousel.html
album/photos.html
beach.jpg
bobcat.jpg
coconuts.jpg
lake.jpg
sunset.jpg

BUCKET developer-b2-quick-start CONTAINS 14 OBJECTS

Process finished with exit code 0

This concludes our review of the sample application's function list_object_keys() and the logic in the conditional block in the sample application's main() function that both calls it and then uses print() statements to output the values returned.

You can explore the remainder of this code on your own. Or you can go to the next lesson in this series.

Video 3: List Browsable URLs: Reviewing Function list_objects_browsable_url()

This next video provides an alternative way of exploring the sample application and reviewing the topics that follow immediately below:

Function list_objects_browsable_url()

Next up is another read-only operation using function list_objects_browsable_url(). This function has similarities to the earlier function list_object_keys(). In fact, it will call list_object_keys(), however it has additional logic to construct and return a valid, browsable URL for each object. The logic in function list_objects_browsable_url() does this by concatenating the bucket's regional endpoint, the bucket name, and each object's key. This can be especially helpful for buckets with access set to Public.

Before looking at the logic in the function, let's first look at the conditional block in the main() function that will invoke it. Following is the full logic of this block.

    # 02 - List Objects formatted as browsable url
    # IF *PUBLIC* BUCKET, PRINT OUTPUTS BROWSABLE URL FOR EACH FILE IN THE BUCKET
    elif len(args) == 1 and (args[0] == '02' or args[0] == '02PUB'):
        # Call function to return list of object 'keys' concatenated into friendly urls
        browsable_urls = list_objects_browsable_url(PUBLIC_BUCKET_NAME, endpoint, b2)
        for url in browsable_urls:
            print(url)

        print('\nBUCKET ', PUBLIC_BUCKET_NAME, ' CONTAINS ', 
              len(browsable_urls), ' OBJECTS')

Following is the conditional else if logic for the second block. This block is executed if an argument is passed (len(args) == 1), and if the value of the input parameter is either "02" or "02PUB" (args[0] == '02' or args[0] == '02PUB').

    # 02 - List Objects formatted as browsable url
    # IF *PUBLIC* BUCKET, PRINT OUTPUTS BROWSABLE URL FOR EACH FILE IN THE BUCKET
    elif len(args) == 1 and (args[0] == '02' or args[0] == '02PUB'):
When logic for this block resolves to true, it executes the sample application function list_objects_browsable_url(). Note that this call is passing in three parameters: PUBLIC_BUCKET_NAME, endpoint, and b2. All three input parameters and how they were created was described above.
    # Call function to return list of object 'keys' concatenated into friendly urls
        browsable_urls = list_objects_browsable_url(PUBLIC_BUCKET_NAME, endpoint, b2)

Before discussing the remaining logic in this conditional block in main(), let's first review the logic inside the function list_objects_browsable_url().

Following are the lines of code you will find in the sample application defining the function list_objects_browsable_url():

# List browsable URLs of the objects in the specified bucket - Useful for *PUBLIC* buckets
def list_objects_browsable_url(bucket, endpoint, b2):
    try:
        bucket_objects = list_object_keys(bucket, b2)

        return_list = []               # create empty list
        for url in bucket_object_keys: # iterate bucket_objects 
            url = "%s/%s/%s" % (endpoint, bucket, url) # format and concatenate 
                                       # strings as valid url 
            return_list.append(url)    # for each item in bucket_objects, 
                                       # append value of 'url' to list
        return return_list             # return list of keys from response

    except ClientError as ce:
        print('error', ce)

Following is sample output from the execution of list_objects_browsable_url() (after print() output in main()):

python sample.py
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/.bzEmpty
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/.bzEmpty
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/
carousel-slider.uiinitiative.com-index.ed866659.css
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/
index.b1995cd6.js
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/
swiper@8.0.3~swiper-bundle.min.css
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/
swiper@8.0.3~swiper-bundle.min.js
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/
vendor.50b6404e.js
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/carousel.html
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/photos.html
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/beach.jpg
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/bobcat.jpg
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/coconuts.jpg
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/lake.jpg
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/sunset.jpg

BUCKET developer-b2-quick-start CONTAINS 14 OBJECTS

Process finished with exit code 0

So, let's step through the logic of the function list_objects_browsable_url(). First, the function's signature specifies three input arguments: bucket, endpoint, and b2. As we saw in the calling code back in main(), the value passed to the bucket will be a string containing the name of the bucket to be referenced. The value passed to endpoint will be a string containing the name of the B2 S3 Compatible API endpoint (that earlier logic in main() retrieved from a constant ENDPOINT in .env). Lastly, the b2 argument must be a reference to a valid B2 Cloud Storage resource service object.

# List browsable URLs of the objects in the specified bucket
# Useful for *PUBLIC* buckets
def list_objects_browsable_url(bucket, endpoint, b2):

Inside the function there are two blocks, try: and except. For our review of the functions in this sample application, we will focus solely on the logic in the try: blocks. For the sample application the except blocks contain only simple print() statements. For details on error handling in boto3, please see the documentation here.

# List browsable URLs of the objects in the specified bucket
# Useful for *PUBLIC* buckets
def list_objects_browsable_url(bucket, endpoint, b2):
    try:
        ...

    except ClientError as ce:
        print('error', ce)

The sample application includes the following among required statements at the top of the application:

from botocore.exceptions import ClientError

Now, let's step through the logic in the try block. First, we take two of the input arguments and invoke the function list_object_keys(bucket, b2) (that was reviewed above), storing the returned iterable collection of object key values in a local variable named bucket_objects. In processing this function call, the logic communicates with the backend Backblaze B2 service and the Backblaze B2 service sends back the iterable collection of ObjectSummary resources, which the logic in function list_object_keys() iterates over and extracts just the key values, which are then returned from function list_object_keys().

        bucket_objects = list_object_keys(bucket, b2)

The intent of the list_objects_browsable_url() function is to return an iterable collection of strings that are valid browsable URLs for each object. The next four lines start by declaring a local variable return_list as an empty list which the next three lines will populate as the logic iterates over the return_list collection. The next line uses a for statement to iterate over the bucket_objects extracting each of those objects' key values. The third line uses the Python formatting "%" operator to concatenate three variable values with literal "/" between each, storing the results in variable url. This operator creates a single string using the %s operator. In execution of this line of code, the strings are replaced in the order of their position in the brackets, wherever there is a %s sign. Then the last line uses the append() method on the list return_list to add each url to the collection.

    return_list = []               # create empty list
    for key in bucket_objects:     # iterate bucket_objects
        url = "%s/%s/%s" % (endpoint, bucket, key)
        return_list.append(url)    # for each item in bucket_objects,
                                   # append value of 'url' to list

The function's processing is now complete and the last line uses the return statement to return back the reference to return_list.

    return return_list             # return list of keys from response

For the sample application, processing now continues back in the main() function conditional block that called list_objects_browsable_url(PUBLIC_BUCKET_NAME, endpoint, b2). The following is the full logic of this calling block.

    # 02 - List Objects formatted as browsable url
    # IF *PUBLIC* BUCKET, PRINT OUTPUTS BROWSABLE URL FOR EACH FILE IN THE BUCKET
    elif len(args) == 1 and (args[0] == '02' or args[0] == '02PUB'):
        # Call function to return list of object 'keys' concatenated into 
        # friendly urls
        browsable_urls = list_objects_browsable_url(PUBLIC_BUCKET_NAME, endpoint, b2)
        for url in browsable_urls:
            print(url)

        print('\nBUCKET ', PUBLIC_BUCKET_NAME, ' CONTAINS ', 
              len(browsable_urls), ' OBJECTS')

Thus, execution of this line of code is now complete. And the variable browsable_urls now contains an iterable collection of valid browsable URLs for each object.

    browsable_urls = list_objects_browsable_url(PUBLIC_BUCKET_NAME, endpoint, b2)

The remainder of the logic here in the conditional block generates es print() output to display the returned results.

The next two lines use a for statement to first iterate over the browsable_urls, extracting from each a reference to each url in the collection. The second line uses a print() statement to display the url value of each item in the collection.

    for url in browsable_urls:
        print(url)

Last, the the conditional block closes with the following print() statement to display the PUBLIC_BUCKET_NAME and a count of the number of keys returned by the call using the logic of len(browsable_urls). These two variable values are concatenated with the literal strings to display output such as:
BUCKET developer-b2-quick-start CONTAINS 14 OBJECTS

    print('\nBUCKET ', PUBLIC_BUCKET_NAME, ' CONTAINS ', 
          len(browsable_urls), ' OBJECTS')

Recapping and repeating, the following is sample output from the execution of list_objects_browsable_url() and the print() statements following its execution in main()):

python sample.py
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/.bzEmpty
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/.bzEmpty
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/
carousel-slider.uiinitiative.com-index.ed866659.css
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/
index.b1995cd6.js
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/
swiper@8.0.3~swiper-bundle.min.css
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/
swiper@8.0.3~swiper-bundle.min.js
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/
vendor.50b6404e.js
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/carousel.html
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/photos.html
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/beach.jpg
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/bobcat.jpg
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/coconuts.jpg
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/lake.jpg
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/sunset.jpg

BUCKET developer-b2-quick-start CONTAINS 14 OBJECTS

Process finished with exit code 0

Try pasting one of the jpg URLs into your browser. Your browser will show the image, since Backblaze B2 returns objects in public buckets via an unsigned HTTPS request.

This concludes our review of the sample application's function list_objects_browsable_url() and the logic in the conditional block in the sample application's main() function that both calls it and then uses print() statements to output the values returned.

For now, you can explore the remainder of this code on your own. We will publish more lessons in the near future.

Feedback

Have a request or idea for the Backblaze S3 Compatible API? Please feel free to contact us at b2feedback@backblaze.com.