Backblaze B2 Quick Start: Using Python With the Backblaze S3 Compatible API
This tutorial provides developers familiar with the Python language with prebuilt code that accesses hosted sample data. In just a few minutes, you’ll get hands-on experience working with Backblaze B2 Cloud Storage. The Backblaze B2 Quick Start includes three options for developers to become proficient with the prebuilt code. First, the open-source code is on GitHub, including helpful commenting in-line. Second, written instructions are provided below for a guided walk-through. Third, video-guided walk-throughs are also embedded below.
KEY TOPICS
1. Overview: Exercise Resources Explained
2. Sample Application: GitHub, Sample Code Structure, Detailed Walk-Throughs of Multiple API Functions
1. OVERVIEW
Hello, and welcome to this Backblaze B2 Quick Start using the Python programming language with the Backblaze S3 Compatible API.
The intended audience are developers who have experience with the Python programming language.
The following instructions provide you with details on using prebuilt Python code that we have shared on GitHub for you to download.
You can get started free of charge by creating a Backblaze B2 account with 10GB of storage at no cost (and downloading up to 1GB/day for free). The instructions below first walk through the prebuilt code on GitHub, making API calls against a prebuilt media application in a pre-staged Backblaze B2 bucket. The API calls against the pre-staged Backblaze B2 bucket are read-only and we have shared the necessary read-only keys in the GitHub project.
Later instructions provide you a guided walk-through of the prebuilt code on GitHub that includes API calls that create buckets, upload files, and delete files and buckets. For this second half of the prebuilt code, you will need to create your own Backblaze account and buckets, which you can do free of charge.
This is part of a comprehensive set of resources that are available to you.
RESOURCES:
- Sample Application: open-source and shared on GitHub for you to download: https://github.com/backblaze-b2-samples/b2-python-s3-sample
- Video Code Walk-throughs of Sample Application: For you to share and rewatch on demand. See videos embedded inline below.
- Hosted Sample Data: A media application with application keys shared for read-only access
- Guided Instructions: These pages that you are currently reading. These instructions guide you on how to download the sample code, run it yourself, and then use the code as you see fit, including incorporating it into your own applications.
These resources initially use our keys, buckets, and sample data. However, 100% of the prebuilt code on GitHub is immediately usable with keys, buckets, and data in your own Backblaze B2 accounts!
Please share these resources with your friends and coworkers.
As you build on B2 Cloud Storage, we'd very much love to hear what you do with it. If you would like to contribute to this project on GitHub, we welcome your participation via pull requests.
Video 1: Connecting to Backblaze B2
This first video provides an alternative way of exploring the sample application and reviewing the topics that follow immediately below:
2. SAMPLE APPLICATION
GitHub
As highlighted above, source code used in these exercises is open-sourced and shared on GitHub for you to download. Download here: https://github.com/backblaze-b2-samples/b2-python-s3-sample
Sample Code Structure (Including .env
File)
This sample application code can be downloaded and executed as is.
Among the GitHub files are two critical files:
- The script file
(
sample.py) - An
.envfile, containing credentials to access a public B2 Cloud Storage bucket.
Typically, GitHub repositories do not contain .env files, but these
credentials are limited to read-only access of public data, so they are safe
to share.
If
you are altogether new to developing with Python, the Python IDE used in our
video walk-through of the code uses PyCharm Community Edition IDE. PyCharm is
not open-source, however the Community Edition is available free for your use
and can be downloaded from here:
https://www.jetbrains.com/pycharm/download/
Hosted Media Application
The Backblaze B2 bucket that we will be using is configured with PUBLIC access. This means that all of the objects in the Backblaze B2 sample data bucket can be downloaded via regular, unsigned HTTP requests. HTTP requests are the basis of the internet and are used by web browsers to request files that browsers display.
The bucket includes a browser photo viewer
application that you can view here:
https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/photos.html
Note, this B2 Cloud Storage bucket contains five .jpg photo files visible at the link above, plus all of the associated HTML, CSS, and JavaScript files for the media viewer application. The JavaScript implementation uses the open-source Swiper project. You can find the Swiper source at: https://github.com/nolimits4web/swiper.
You’ll find it useful to open sample.py in a code editor so you can refer
to it as you read the following explanation.
Function
get_b2_resource()
In the sample application, the critical logic showing how to get a connection
to B2 Cloud Storage is in the function named get_b2_resource(). The following are the lines of code you will find in the sample application
defining the function get_b2_resource():
# Return a boto3 resource object for B2 service
def get_b2_resource(endpoint, key_id, application_key):
b2 = boto3.resource(service_name='s3',
endpoint_url=endpoint, # Backblaze endpoint
aws_access_key_id=key_id, # Backblaze keyID
aws_secret_access_key=application_key,
# Backblaze applicationKey
config=Config(
signature_version='s3v4'))
return b2
As you can see in the code, the sample application uses the open-source
boto3 library. On execution the function returns a
boto3 service resource. In the code the value returned is stored in a
variable named b2. Referencing this
b2 variable you can then call on the B2 Cloud Storage service
actions, sub-resources and collections. For a full set of supported calls
supported by the B2 Cloud Storage service see the
documentation
here.
The sample application includes the following among required import statements at the top of the application:
import boto3
For details on downloading and installing the current version of boto3, please see: https://pypi.org/project/boto3/
Variations of this logic can be deployed wherever you run your code. This might be on local machines, running in browsers, or running on servers. Regardless you're going to be storing and interacting with files that are stored out in the cloud on the Backblaze B2 service.
The get_b2_resource() function passes five parameters to
boto3.resource():
service_nameendpoint_urlaws_access_key_idaws_secret_access_keyconfig
The first named parameter, service_name, selected boto3’s
‘s3’ service.
Note: Although the value being passed is the literal string 's3',
all API calls will go to the Backblaze B2 service via the Backblaze S3
Compatible API.
API request routing and authentication are governed by the remaining four parameters and settings in the B2 Cloud Storage bucket:
endpoint_url=endpointaws_access_key_id=keyID-
aws_secret_access_key=applicationKey -
config
The middle two parameters both start with the prefix aws_,
however, the values being passed apply to the Backblaze B2 service.
Overriding the default Amazon S3 endpoint URL allows us to use boto3 to connect to Backblaze B2.
In the sample code, the values being passed via 3 of these parameters
are retrieved from the .env file.
The
names of all parameters are hardcoded in the boto3 library. The middle two both
start with the prefix aws_ however the values being passed are from
the Backblaze B2 service.
CONSTANTS
AND .env FILE
The following are the the lines of code you will find in the sample application
file .env():
# B2 API endpoint for buckets with sample data ENDPOINT='https://s3.us-west-002.backblazeb2.com' # Following 2 application key pairs provide read-only access # to buckets with sample data # Bucket with sample data and PUBLIC access KEY_ID_RO='0027464dd94917b0000000001' APPLICATION_KEY_RO='K002WU+TkHXkksxIqI6IDa/X7dsN9Cw' # Bucket with sample data and PRIVATE access KEY_ID_PRIVATE_RO='0027464dd94917b0000000002' APPLICATION_KEY_PRIVATE_RO='K002ckrkS/KpaRA9IFzC3xyIn79ALw4' # Variables below for functions that require write access! # You must set these values using your own Backblaze account # 1. Retrieve B2 API Endpoint for region containing your Bucket # 2. Create Key Pair in Backblaze Console # Direct Link here to "App Keys" page https://secure.backblaze.com/app_keys.htm # In Backblaze console, select "App Keys" on left-side nav (3rd up from bottom) # Then select "Add a New Application Key" then "Read and Write" re "Type of Access" # In Backblaze console, values are labeled as keyID and applicationKey respectively ENDPOINT_URL_YOUR_BUCKET='<ENTER YOUR B2 API ENDPOINT HERE!>' KEY_ID_YOUR_ACCOUNT='<ENTER YOUR keyID HERE!>' APPLICATION_KEY_YOUR_ACCOUNT='<ENTER YOUR applicationKey HERE!>'
These constants are stored separately in the .env file
because it is best practice to never check key values and other secrets into
a repository. We are making an exception in this case in order for you to
have a very fast development startup experience. In your own production
applications you should add .env to .gitignore to avoid checking in the .env
file. Please note that in .env the names of both constants
for the key values end with an _ro for “read only” to indicate
that these keys are related to read-only access. Please also note the
comments in the .env file that identify how the key variables
are labeled in the Backblaze console: keyID and applicationKey. When you
use this code against buckets in your own account you will generate your own
keys via the “App Keys” section under “Account” in the Backblaze console.
When get_b2_resource() is called it invokes the boto3.resource()
method which returns a reference to the Backblaze B2 ServiceResource object. With
the Backblaze B2 ServiceResource object, your code can now call additional actions and
sub-resources defined on it in boto3. For details on actions and
sub-resources see the
Boto3 Docs. For the list of S3 calls supported by B2 Cloud
Storage, see the
docs for
the Backblaze S3 Compatible API.
This
script is designed to successfully execute without any parameters. In this
sample application code, the
main() function calls get_b2_resource(), then, by
default, calls list_object_keys().
When executed, the script prints the names, or “keys,” of the objects in the
specified bucket. By default the bucket referenced is set on the constant named
PUBLIC_BUCKET_NAME. See “Hosted media application” above for
details of the sample bucket.
Among the keys displayed, you will see the following five photos:
beach.jpgbobcat.jpgcoconuts.jpglake.jpgsunset.jpg
Video 2: Reviewing Two functions: Sample Application
main() and
list_object_keys()
This next video provides an alternative way of exploring the sample application and reviewing the topics that follow immediately below:
Function
main()
The following are a sample of the lines of code you will find in the sample application
defining the function main():
"""
Python main()
Basic execution setup
Then conditional blocks executing based on command-line arguments passed as input.
"""
def main():
args = sys.argv[1:] # retrieve command-line arguments passed to the script
load_dotenv() # load environment variables from file .env
# get environment variables from file .env
endpoint = os.getenv("ENDPOINT") # Backblaze endpoint
key_id_ro = os.getenv("KEY_ID_RO") # Backblaze keyID
application_key_ro = os.getenv("APPLICATION_KEY_RO") # Backblaze applicationKey
# Call function to return reference to B2 service
b2 = get_b2_resource(endpoint, key_id_ro, application_key_ro)
client = boto3.client(service_name='s3',
endpoint_url=endpoint,
aws_access_key_id=key_id_ro,
aws_secret_access_key=application_key_ro)
# pyboto3 provides Pythonic Interface for typehint for autocomplete in pycharm
""" :type : pyboto3.s3 """
# get environment variables from file .env
# Backblaze keyID
key_id_private_ro = os.getenv("KEY_ID_PRIVATE_RO")
# Backblaze applicationKey
application_key_private_ro = os.getenv("APPLICATION_KEY_PRIVATE_RO")
# Call function to return reference to B2 service using a second set of keys
b2_private = get_b2_resource(endpoint,
key_id_private_ro, application_key_private_ro)
# 01 - list_object_keys
if len(args) == 0 or (len(args) == 1 and args[0] == '01'):
# Call function to return list of object 'keys'
bucket_object_keys = list_object_keys(PUBLIC_BUCKET_NAME, b2)
for key in bucket_object_keys:
print(key)
print('\nBUCKET ', PUBLIC_BUCKET_NAME, ' CONTAINS ',
len(bucket_object_keys), ' OBJECTS')
# 02 - List Objects formatted as browsable url
# IF *PUBLIC* BUCKET, PRINT OUTPUTS BROWSABLE URL FOR EACH FILE IN THE BUCKET
elif len(args) == 1 and (args[0] == '02' or args[0] == '02PUB'):
# Call function to return list of object 'keys' concatenated into
# friendly urls
browsable_urls = list_objects_browsable_url(PUBLIC_BUCKET_NAME, endpoint, b2)
for key in browsable_urls:
print(key)
print('\nBUCKET ', PUBLIC_BUCKET_NAME, ' CONTAINS ',
len(browsable_urls), ' OBJECTS')
In Python programs, the main() function is both the
top-level environment and the starting point of logic each time the program
is executed. When the program is run, the Python interpreter runs the
code sequentially.
The first statement in main() reflects that the sample
application is architected to optionally take parameters as input. The following
line of code retrieves the command-line arguments passed to the script and
stores them in the variable args. In later lines of code
there are several conditional blocks testing for various expected
command-line arguments expected to be passed as input.
args = sys.argv[1:] # retrieve command-line arguments passed to the script
The sample application includes the following among required import statements at the top of the application:
import sys
The next lines of code load the values from the .env file. How this application is using the .env file was discussed
earlier under the heading "CONSTANTS AND .env FILE" above. The
last three lines below load each of the string values
stored in the .env file into local variables:
load_dotenv() # load environment variables from file .env
# get environment variables from file .env
endpoint = os.getenv("ENDPOINT") # Backblaze endpoint
key_id_ro = os.getenv("KEY_ID_RO") # Backblaze keyID
application_key_ro = os.getenv("APPLICATION_KEY_RO") # Backblaze applicationKey
The sample application includes the following among required import statements at the top of the application:
import boto3 # REQUIRED! - Details here: https://pypi.org/project/boto3/ from botocore.exceptions import ClientError from botocore.config import Config from dotenv import load_dotenv # Project Must install Python Package: python-dotenv import os import sys
The next line of code calls the function
get_b2_resource(). The logic inside this function was discussed
earlier under the heading "Function
get_b2_resource()" above. Note that in invoking this function,
this line of code passes as input the values retrieved from the .env
file. This function returns a
resource object for the B2 Cloud Storage service. The reference to this resource
object is stored in variable b2 which will be referenced in
subsequent lines of code here in the sample application.
# Call function to return reference to B2 service
b2 = get_b2_resource(endpoint, key_id_ro, application_key_ro)
There are additional setup statements at the top of main().
They will be discussed later when we review the code that use them.
The remainder of the logic in the main() function is in
conditional blocks. With each execution of the sample program,
only one conditional block will execute. In turn, each conditional
block executes a call to the B2 Cloud Storage service.
Since each conditional block executes a sample application function, the
remainder of the logic in the main() function will be reviewed
below in conjunction with the description of the functions that they call.
Function
list_object_keys()
First up is a read-only operation to
list_object_keys().
For the sample application, the default case when no arguments are passed
is the execution of the first conditional block in main()
function. Following is the full logic of this first block.
# 01 - list_object_keys
if len(args) == 1 and args[0] == '01':
# Call function to return list of object 'keys'
bucket_object_keys = list_object_keys(PUBLIC_BUCKET_NAME, b2)
for key in bucket_object_keys:
print(key)
print('\nBUCKET ', PUBLIC_BUCKET_NAME, ' CONTAINS ',
len(bucket_object_keys), ' OBJECTS')
Following is the conditional logic for the first block. This is the
default case if no arguments are passed (len(args) == 0), or
if input parameter of "01" is passed (if (len(args) == 1 and args[0] == '01')). When logic for this block resolves to true, it will execute the sample
application function named
list_object_keys().
# 01 - list_object_keys
if len(args) == 1 and args[0] == '01':
When this block executes, it in turn executes the following call on the
function list_object_keys(). Note that this call is passing
in two parameters: PUBLIC_BUCKET_NAME and b2. The input parameter b2 and how it was created was described
above.
# Call function to return list of object 'keys'
bucket_object_keys = list_object_keys(PUBLIC_BUCKET_NAME, b2)
PUBLIC_BUCKET_NAME is a constant defined at the top of
the sample application as follows:
# Bucket with Sample Data **PUBLIC**
PUBLIC_BUCKET_NAME = 'developer-b2-quick-start2'
Before discussing remaining logic in this conditional block in
main(), let's first review the logic inside function
list_object_keys().
# List the keys of the objects in the specified bucket
def list_object_keys(bucket, b2):
try:
response = b2.Bucket(bucket).objects.all()
return_list = [] # create empty list
for object in response: # iterate over response
return_list.append(object.key) # for each item in response,
# append object.key to list
return return_list # return list of keys from response
except ClientError as ce:
print('error', ce)
The following is sample output from execution of list_object_keys()
(after print() output in main()):
"C:\temp\B2_Python_Quick_Start\python.exe" C:/SAMPLES/B2_Python_Quick_Start/sample.py album/.bzEmpty album/assets/.bzEmpty album/assets/carousel-slider.uiinitiative.com-index.ed866659.css album/assets/index.b1995cd6.js album/assets/swiper@8.0.3~swiper-bundle.min.css album/assets/swiper@8.0.3~swiper-bundle.min.js album/assets/vendor.50b6404e.js album/carousel.html album/photos.html beach.jpg bobcat.jpg coconuts.jpg lake.jpg sunset.jpg BUCKET developer-b2-quick-start CONTAINS 14 OBJECTS Process finished with exit code 0
So, let's step through the logic of the function list_object_keys().
First, the function's signature specifies two input arguments, bucket
and b2. As we saw in the calling code back in
main(), the value passed to the bucket argument will be a string
containing the name of the bucket to be referenced. And the b2
argument must be a reference to a valid Backblaze B2 resource service object.
# List the keys of the objects in the specified bucket def list_object_keys(bucket, b2):
Inside the function there are two blocks, try: and
except. For our review of the functions in this sample
application, we will focus solely on the logic in the try:
blocks. For the sample application, the except blocks
contain only simple print() statements. For details on error
handling in boto3, please see the
documentation here.
# List the keys of the objects in the specified bucket
def list_object_keys(bucket, b2):
try:
...
except ClientError as ce:
print('error', ce)
The
sample application includes the following among required statements at the top
of the application:from botocore.exceptions import ClientError.
Now let's step through the logic in the try block. First, we take the two input arguments and using method-chaining syntax,
return back an iterable collection of
ObjectSummary resources and store them in a local variable named response. Reading this first line from
left to right, we take the b2 input argument and call on it the constructor for
the bucket sub-resource, passing in as input the bucket
input argument. With that the chain now has reference to a
B2 Cloud Storage bucket
resource object. Continuing the chain, the logic next references
the bucket resource's available objects collection and calls on
it the function all(). In processing this chain, the
logic communicates with the backend Backblaze B2 service and the Backblaze B2 service
sends back the iterable collection of
ObjectSummary resources which our logic now stores locally in variable response.
response = b2.Bucket(bucket).objects.all()
The intent of the list_object_keys() function is to return back to the caller an iterable
collection of object
key values. The next three lines start by declaring a local variable
return_list as an empty list which the next two lines will populate
as the logic iterates over the response collection. The next
line uses a for statement to iterate over the response,
extracting each object. The third line uses the append() method on the
list return_list to add the key value on each object to the collection.
return_list = [] # create empty list
for object in response: # iterate over response
return_list.append(object.key) # for each item in response,
# append object.key to list
The function's processing is now complete and the last line uses the
return statement to return back the reference to return_list.
return return_list # return list of objects from response
For the sample application, processing now continues back in the main()
function conditional block that called list_object_keys(PUBLIC_BUCKET_NAME, b2). The
following is the full logic of this
calling block.
# 01 - list_object_keys
if len(args) == 1 and args[0] == '01':
# Call function to return list of object 'keys'
bucket_object_keys = list_object_keys(PUBLIC_BUCKET_NAME, b2)
for key in bucket_object_keys:
print(key)
print('\nBUCKET ', PUBLIC_BUCKET_NAME, ' CONTAINS ',
len(bucket_object_keys), ' OBJECTS')
Thus, the execution of this line of code is now complete. And the
variable bucket_object_keys now contains an iterable
collection of object
key values.
bucket_object_keys = list_object_keys(PUBLIC_BUCKET_NAME, b2)
The remainder of the logic here in the conditional block generates
print() output to display the returned results.
The next two lines use a for statement to first iterate over the
bucket_object_keys
extracting from each a reference to each object in the collection. The
second line uses a print() statement to display the
key
of each object in the collection.
for key in bucket_object_keys:
print(key)
Lastly, the conditional block closes with the following print()
statement to display the PUBLIC_BUCKET_NAME and a count of the
number of keys returned by the call using logic of
len(bucket_object_keys). These two variable values are
concatenated with 3 literal strings to display output such as:
BUCKET developer-b2-quick-start CONTAINS 14 OBJECTS
print('\nBUCKET ', PUBLIC_BUCKET_NAME, ' CONTAINS ',
len(bucket_object_keys), ' OBJECTS')
Recapping and repeating, the following is sample output from the execution of
list_object_keys()
and the print() statements following its execution in main()):
python sample.py album/.bzEmpty album/assets/.bzEmpty album/assets/carousel-slider.uiinitiative.com-index.ed866659.css album/assets/index.b1995cd6.js album/assets/swiper@8.0.3~swiper-bundle.min.css album/assets/swiper@8.0.3~swiper-bundle.min.js album/assets/vendor.50b6404e.js album/carousel.html album/photos.html beach.jpg bobcat.jpg coconuts.jpg lake.jpg sunset.jpg BUCKET developer-b2-quick-start CONTAINS 14 OBJECTS Process finished with exit code 0
This concludes our review of the sample application's function
list_object_keys() and the logic in the conditional block in the sample
application's main()
function that both calls it and then uses print() statements to
output the values returned.
You can explore the remainder of this code on your own. Or you can go to the next lesson in this series.
Video 3: List Browsable URLs: Reviewing Function
list_objects_browsable_url()
This next video provides an alternative way of exploring the sample application and reviewing the topics that follow immediately below:
Function
list_objects_browsable_url()
Next up is another read-only operation using function
list_objects_browsable_url(). This function has
similarities to the earlier function
list_object_keys(). In fact, it will call
list_object_keys(), however it has additional logic to construct and
return a valid, browsable URL for each object. The logic in function
list_objects_browsable_url() does this by concatenating the bucket's
regional endpoint, the bucket name, and each object's key. This can be
especially helpful for buckets with access set to Public.
Before looking at the logic in the function, let's first look at the
conditional block in the main() function that will invoke it. Following is the full logic of this block.
# 02 - List Objects formatted as browsable url
# IF *PUBLIC* BUCKET, PRINT OUTPUTS BROWSABLE URL FOR EACH FILE IN THE BUCKET
elif len(args) == 1 and (args[0] == '02' or args[0] == '02PUB'):
# Call function to return list of object 'keys' concatenated into friendly urls
browsable_urls = list_objects_browsable_url(PUBLIC_BUCKET_NAME, endpoint, b2)
for url in browsable_urls:
print(url)
print('\nBUCKET ', PUBLIC_BUCKET_NAME, ' CONTAINS ',
len(browsable_urls), ' OBJECTS')
Following is the conditional else if logic for the second block. This
block is executed if an argument is passed (len(args) == 1),
and if
the value of the input parameter is either "02" or "02PUB" (args[0] == '02' or args[0] == '02PUB').
# 02 - List Objects formatted as browsable url
# IF *PUBLIC* BUCKET, PRINT OUTPUTS BROWSABLE URL FOR EACH FILE IN THE BUCKET
elif len(args) == 1 and (args[0] == '02' or args[0] == '02PUB'):
When logic for this block resolves to true, it executes the sample
application function
list_objects_browsable_url(). Note that this call is passing
in three parameters: PUBLIC_BUCKET_NAME, endpoint, and b2. All three input parameters and how they were created was described
above.
# Call function to return list of object 'keys' concatenated into friendly urls
browsable_urls = list_objects_browsable_url(PUBLIC_BUCKET_NAME, endpoint, b2)
Before discussing the remaining logic in this conditional block in
main(), let's first review the logic inside the function
list_objects_browsable_url().
Following are the lines of code you will find in the sample application
defining the function list_objects_browsable_url():
# List browsable URLs of the objects in the specified bucket - Useful for *PUBLIC* buckets
def list_objects_browsable_url(bucket, endpoint, b2):
try:
bucket_objects = list_object_keys(bucket, b2)
return_list = [] # create empty list
for url in bucket_object_keys: # iterate bucket_objects
url = "%s/%s/%s" % (endpoint, bucket, url) # format and concatenate
# strings as valid url
return_list.append(url) # for each item in bucket_objects,
# append value of 'url' to list
return return_list # return list of keys from response
except ClientError as ce:
print('error', ce)
Following is sample output from the execution of
list_objects_browsable_url()
(after print() output in main()):
python sample.py https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/.bzEmpty https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/.bzEmpty https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/ carousel-slider.uiinitiative.com-index.ed866659.css https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/ index.b1995cd6.js https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/ swiper@8.0.3~swiper-bundle.min.css https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/ swiper@8.0.3~swiper-bundle.min.js https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/ vendor.50b6404e.js https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/carousel.html https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/photos.html https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/beach.jpg https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/bobcat.jpg https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/coconuts.jpg https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/lake.jpg https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/sunset.jpg BUCKET developer-b2-quick-start CONTAINS 14 OBJECTS Process finished with exit code 0
So, let's step through the logic of the function
list_objects_browsable_url(). First, the function's signature
specifies three input arguments: bucket, endpoint,
and b2. As we saw in the calling code back in
main(), the value passed to the bucket will be a string
containing the name of the bucket to be referenced. The value
passed to endpoint will be a string containing the name of the
B2 S3 Compatible API endpoint (that earlier logic in main() retrieved from a
constant ENDPOINT in .env). Lastly, the b2
argument must be a reference to a valid B2 Cloud Storage resource service object.
# List browsable URLs of the objects in the specified bucket # Useful for *PUBLIC* buckets def list_objects_browsable_url(bucket, endpoint, b2):
Inside the function there are two blocks, try: and
except. For our review of the functions in this sample
application, we will focus solely on the logic in the try:
blocks. For the sample application the except blocks
contain only simple print() statements. For details on error
handling in boto3, please see the
documentation here.
# List browsable URLs of the objects in the specified bucket
# Useful for *PUBLIC* buckets
def list_objects_browsable_url(bucket, endpoint, b2):
try:
...
except ClientError as ce:
print('error', ce)
The sample application includes the following among required statements at the top of the application:
from botocore.exceptions import ClientError
Now, let's step through the logic in the try block. First, we take two of the input arguments and invoke the function
list_object_keys(bucket, b2)
(that was reviewed above), storing the returned iterable collection of
object
key values in a local variable named bucket_objects. In processing this
function call, the
logic communicates with the backend Backblaze B2 service and the Backblaze B2 service
sends back the iterable collection of
ObjectSummary resources, which the logic in function list_object_keys()
iterates over and extracts just the key values, which are then
returned from function list_object_keys().
bucket_objects = list_object_keys(bucket, b2)
The intent of the list_objects_browsable_url() function is to return an iterable
collection of strings that are valid browsable URLs for each object. The next
four lines start by declaring a local variable
return_list as an empty list which the next three lines will populate
as the logic iterates over the
return_list collection. The next
line uses a for statement to iterate over the bucket_objects extracting each of those objects'
key values. The third line uses the Python formatting "%"
operator to concatenate three variable values with literal "/" between
each, storing the results in variable url. This operator
creates a single string using the %s operator. In execution
of this line of code, the strings are replaced in the order of their position in
the brackets, wherever there is a %s sign. Then the last
line uses the append() method on the
list return_list to add each url to the collection.
return_list = [] # create empty list
for key in bucket_objects: # iterate bucket_objects
url = "%s/%s/%s" % (endpoint, bucket, key)
return_list.append(url) # for each item in bucket_objects,
# append value of 'url' to list
The function's processing is now complete and the last line uses the
return statement to return back the reference to return_list.
return return_list # return list of keys from response
For the sample application, processing now continues back in the main()
function conditional block that called list_objects_browsable_url(PUBLIC_BUCKET_NAME, endpoint, b2). The
following is the full logic of this
calling block.
# 02 - List Objects formatted as browsable url
# IF *PUBLIC* BUCKET, PRINT OUTPUTS BROWSABLE URL FOR EACH FILE IN THE BUCKET
elif len(args) == 1 and (args[0] == '02' or args[0] == '02PUB'):
# Call function to return list of object 'keys' concatenated into
# friendly urls
browsable_urls = list_objects_browsable_url(PUBLIC_BUCKET_NAME, endpoint, b2)
for url in browsable_urls:
print(url)
print('\nBUCKET ', PUBLIC_BUCKET_NAME, ' CONTAINS ',
len(browsable_urls), ' OBJECTS')
Thus, execution of this line of code is now complete. And the
variable browsable_urls now contains an iterable collection of
valid browsable URLs for each object.
browsable_urls = list_objects_browsable_url(PUBLIC_BUCKET_NAME, endpoint, b2)
The remainder of the logic here in the conditional block generates es
print() output to display the returned results.
The next two lines use a for statement to first iterate over the
browsable_urls,
extracting from each a reference to each url in the collection. The
second line uses a print() statement to display the url
value
of each item in the collection.
for url in browsable_urls:
print(url)
Last, the the conditional block closes with the following print()
statement to display the PUBLIC_BUCKET_NAME and a count of the
number of keys returned by the call using the logic of
len(browsable_urls). These two variable values are
concatenated with the literal strings to display output such as:
BUCKET developer-b2-quick-start CONTAINS 14 OBJECTS
print('\nBUCKET ', PUBLIC_BUCKET_NAME, ' CONTAINS ',
len(browsable_urls), ' OBJECTS')
Recapping and repeating, the following is sample output from the execution of
list_objects_browsable_url()
and the print() statements following its execution in main()):
python sample.py https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/.bzEmpty https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/.bzEmpty https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/ carousel-slider.uiinitiative.com-index.ed866659.css https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/ index.b1995cd6.js https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/ swiper@8.0.3~swiper-bundle.min.css https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/ swiper@8.0.3~swiper-bundle.min.js https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/assets/ vendor.50b6404e.js https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/carousel.html https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/album/photos.html https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/beach.jpg https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/bobcat.jpg https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/coconuts.jpg https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/lake.jpg https://s3.us-west-002.backblazeb2.com/developer-b2-quick-start/sunset.jpg BUCKET developer-b2-quick-start CONTAINS 14 OBJECTS Process finished with exit code 0
Try pasting one of the jpg URLs into your browser. Your browser will show the image, since Backblaze B2 returns objects in public buckets via an unsigned HTTPS request.
This concludes our review of the sample application's function
list_objects_browsable_url() and the logic in the conditional block in the sample
application's main()
function that both calls it and then uses print() statements to
output the values returned.
For now, you can explore the remainder of this code on your own. We will publish more lessons in the near future.
Feedback
Have a request or idea for the Backblaze S3 Compatible API? Please feel free to contact us at b2feedback@backblaze.com.