Main and List Object Keys Functions

This next video provides an alternative way of exploring the sample application and reviewing the topics that follow immediately below:

Function main()

The following are a sample of the lines of code you will find in the sample application defining the function main():

"""
Python main() 

Basic execution setup
Then conditional blocks executing based on command-line arguments passed as input.
"""
def main():
    args = sys.argv[1:]  # retrieve command-line arguments passed to the script

    load_dotenv()                   # load environment variables from file .env

    # get environment variables from file .env
    endpoint = os.getenv("ENDPOINT")  # Backblaze endpoint
    key_id_ro = os.getenv("KEY_ID_RO")  # Backblaze keyID
    application_key_ro = os.getenv("APPLICATION_KEY_RO") # Backblaze applicationKey

    # Call function to return reference to B2 service
    b2 = get_b2_resource(endpoint, key_id_ro, application_key_ro)

    client = boto3.client(service_name='s3',
                          endpoint_url=endpoint,
                          aws_access_key_id=key_id_ro,
                          aws_secret_access_key=application_key_ro)
    # pyboto3 provides Pythonic Interface for typehint for autocomplete in pycharm
    """ :type : pyboto3.s3 """

    # get environment variables from file .env
    # Backblaze keyID
    key_id_private_ro = os.getenv("KEY_ID_PRIVATE_RO")
    # Backblaze applicationKey
    application_key_private_ro = os.getenv("APPLICATION_KEY_PRIVATE_RO")

    # Call function to return reference to B2 service using a second set of keys
    b2_private = get_b2_resource(endpoint, 
                                 key_id_private_ro, application_key_private_ro)

    # 01 - list_object_keys
    if len(args) == 0 or (len(args) == 1 and args[0] == '01'):
        # Call function to return list of object 'keys'
        bucket_object_keys = list_object_keys(PUBLIC_BUCKET_NAME, b2)
        for key in bucket_object_keys:
            print(key)

        print('\nBUCKET ', PUBLIC_BUCKET_NAME, ' CONTAINS ', 
              len(bucket_object_keys), ' OBJECTS')

    # 02 - List Objects formatted as browsable url
    # IF *PUBLIC* BUCKET, PRINT OUTPUTS BROWSABLE URL FOR EACH FILE IN THE BUCKET
    elif len(args) == 1 and (args[0] == '02' or args[0] == '02PUB'):
        # Call function to return list of object 'keys' concatenated into 
        # friendly urls
        browsable_urls = list_objects_browsable_url(PUBLIC_BUCKET_NAME, endpoint, b2)
        for key in browsable_urls:
            print(key)

        print('\nBUCKET ', PUBLIC_BUCKET_NAME, ' CONTAINS ', 
              len(browsable_urls), ' OBJECTS')

In Python programs, the main() function is both the top-level environment and the starting point of logic each time the program is executed. When the program is run, the Python interpreter runs the code sequentially.

The first statement in main() reflects that the sample application is architected to optionally take parameters as input. The following line of code retrieves the command-line arguments passed to the script and stores them in the variable args. In later lines of code there are several conditional blocks testing for various expected command-line arguments expected to be passed as input.

args = sys.argv[1:]  # retrieve command-line arguments passed to the script

The sample application includes the following among required import statements at the top of the application:

import sys

The next lines of code load the values from the .env file. How this application is using the .env file was discussed earlier under the heading "CONSTANTS AND .env FILE" above. The last three lines below load each of the string values stored in the .env file into local variables:

load_dotenv()   # load environment variables from file .env

    # get environment variables from file .env
    endpoint = os.getenv("ENDPOINT")  # Backblaze endpoint
    key_id_ro = os.getenv("KEY_ID_RO")  # Backblaze keyID
    application_key_ro = os.getenv("APPLICATION_KEY_RO") # Backblaze applicationKey

The sample application includes the following among required import statements at the top of the application:

import boto3  # REQUIRED! - Details here: https://pypi.org/project/boto3/
from botocore.exceptions import ClientError
from botocore.config import Config
from dotenv import load_dotenv  # Project Must install Python Package:  python-dotenv
import os
import sys

The next line of code calls the function get_b2_resource(). The logic inside this function was discussed earlier under the heading "Function get_b2_resource()" above. Note that in invoking this function, this line of code passes as input the values retrieved from the .env file. This function returns a resource object for the Backblaze B2 Cloud Storage service. The reference to this resource object is stored in variable b2 which will be referenced in subsequent lines of code here in the sample application.

# Call function to return reference to B2 service
    b2 = get_b2_resource(endpoint, key_id_ro, application_key_ro)

There are additional setup statements at the top of main(). They will be discussed later when we review the code that use them.

The remainder of the logic in the main() function is in conditional blocks. With each execution of the sample program, only one conditional block will execute. In turn, each conditional block executes a call to the Backblaze B2 service.

Since each conditional block executes a sample application function, the remainder of the logic in the main() function will be reviewed below in conjunction with the description of the functions that they call.

Function list_object_keys()

First up is a read-only operation to list_object_keys().

For the sample application, the default case when no arguments are passed is the execution of the first conditional block in main() function. Following is the full logic of this first block.

# 01 - list_object_keys
    if len(args) == 1 and args[0] == '01':
        # Call function to return list of object 'keys'
        bucket_object_keys = list_object_keys(PUBLIC_BUCKET_NAME, b2)
        for key in bucket_object_keys:
            print(key)

        print('\nBUCKET ', PUBLIC_BUCKET_NAME, ' CONTAINS ', 
              len(bucket_object_keys), ' OBJECTS')

Following is the conditional logic for the first block. This is the default case if no arguments are passed (len(args) == 0), or if input parameter of "01" is passed (if (len(args) == 1 and args[0] == '01')). When logic for this block resolves to true, it will execute the sample application function named list_object_keys().

# 01 - list_object_keys
    if len(args) == 1 and args[0] == '01':

When this block executes, it in turn executes the following call on the function list_object_keys(). Note that this call is passing in two parameters: PUBLIC_BUCKET_NAME and b2. The input parameter b2 and how it was created was described above.

# Call function to return list of object 'keys'
    bucket_object_keys = list_object_keys(PUBLIC_BUCKET_NAME, b2)

PUBLIC_BUCKET_NAME is a constant defined at the top of the sample application as follows:

# Bucket with Sample Data **PUBLIC**
    PUBLIC_BUCKET_NAME = 'developer-b2-quick-start2'

Before discussing remaining logic in this conditional block in main(), let's first review the logic inside function list_object_keys().

# List the keys of the objects in the specified bucket 
def list_object_keys(bucket, b2):
    try:
        response = b2.Bucket(bucket).objects.all()

        return_list = []               # create empty list
        for object in response:        # iterate over response
            return_list.append(object.key) # for each item in response,
                                       # append object.key to list   
        return return_list             # return list of keys from response 

    except ClientError as ce:
        print('error', ce)

The following is sample output from execution of list_object_keys() (after print() output in main()):

"C:\temp\B2_Python_Quick_Start\python.exe" C:/SAMPLES/B2_Python_Quick_Start/sample.py 
album/.bzEmpty
album/assets/.bzEmpty
album/assets/carousel-slider.uiinitiative.com-index.ed866659.css
album/assets/index.b1995cd6.js
album/assets/[email protected]~swiper-bundle.min.css
album/assets/[email protected]~swiper-bundle.min.js
album/assets/vendor.50b6404e.js
album/carousel.html
album/photos.html
beach.jpg
bobcat.jpg
coconuts.jpg
lake.jpg
sunset.jpg

BUCKET developer-b2-quick-start CONTAINS 14 OBJECTS

Process finished with exit code 0

So, let's step through the logic of the function list_object_keys(). First, the function's signature specifies two input arguments, bucket and b2. As we saw in the calling code back in main(), the value passed to the bucket argument will be a string containing the name of the bucket to be referenced. And the b2 argument must be a reference to a valid Backblaze B2 resource service object.

# List the keys of the objects in the specified bucket 
def list_object_keys(bucket, b2):

Inside the function there are two blocks, try: and except. For our review of the functions in this sample application, we will focus solely on the logic in the try: blocks. For the sample application, the except blocks contain only simple print() statements. For details on error handling in boto3, please see the documentation here.

# List the keys of the objects in the specified bucket 
def list_object_keys(bucket, b2):
    try:
        ...

    except ClientError as ce:
        print('error', ce)

The sample application includes the following among required statements at the top of the application:

from botocore.exceptions import ClientError

Now let's step through the logic in the try block. First, we take the two input arguments and using method-chaining syntax, return back an iterable collection of ObjectSummary resources and store them in a local variable named response. Reading this first line from left to right, we take the b2 input argument and call on it the constructor for the bucket sub-resource, passing in as input the bucket input argument. With that the chain now has reference to a Backblaze B2 bucket resource object. Continuing the chain, the logic next references the bucket resource's available objects collection and calls on it the function all(). In processing this chain, the logic communicates with the backend Backblaze B2 service and the Backblaze B2 service sends back the iterable collection of ObjectSummary resources which our logic now stores locally in variable response.

        response = b2.Bucket(bucket).objects.all()

The intent of the list_object_keys() function is to return back to the caller an iterable collection of object key values. The next three lines start by declaring a local variable return_list as an empty list which the next two lines will populate as the logic iterates over the response collection. The next line uses a for statement to iterate over the response, extracting each object. The third line uses the append() method on the list return_list to add the key value on each object to the collection.

 return_list = []               # create empty list
    for object in response:        # iterate over response
        return_list.append(object.key) # for each item in response,
                                   # append object.key to list 

The function's processing is now complete and the last line uses the return statement to return back the reference to return_list.

    return return_list             # return list of objects from response

For the sample application, processing now continues back in the main() function conditional block that called list_object_keys(PUBLIC_BUCKET_NAME, b2). The following is the full logic of this calling block.

# 01 - list_object_keys
    if len(args) == 1 and args[0] == '01':
        # Call function to return list of object 'keys'
        bucket_object_keys = list_object_keys(PUBLIC_BUCKET_NAME, b2)
        for key in bucket_object_keys:
            print(key)

        print('\nBUCKET ', PUBLIC_BUCKET_NAME, ' CONTAINS ', 
              len(bucket_object_keys), ' OBJECTS')

Thus, the execution of this line of code is now complete. And the variable bucket_object_keys now contains an iterable collection of object key values.

    bucket_object_keys = list_object_keys(PUBLIC_BUCKET_NAME, b2)

The remainder of the logic here in the conditional block generates print() the returned results.

The next two lines use a for statement to first iterate over the bucket_object_keys extracting from each a reference to each object in the collection. The second line uses a print() statement to display the key of each object in the collection.

for key in bucket_object_keys:
        print(key)

Lastly, the conditional block closes with the following print() statement to display the PUBLIC_BUCKET_NAME and a count of the number of keys returned by the call using logic of len(bucket_object_keys). These two variable values are concatenated with 3 literal strings to display output such as:

BUCKET developer-b2-quick-start CONTAINS 14 OBJECTS

    print('\nBUCKET ', PUBLIC_BUCKET_NAME, ' CONTAINS ', 
          len(bucket_object_keys), ' OBJECTS')

Recapping and repeating, the following is sample output from the execution of list_object_keys() and the print() statements following its execution in main()):

python sample.py 
album/.bzEmpty
album/assets/.bzEmpty
album/assets/carousel-slider.uiinitiative.com-index.ed866659.css
album/assets/index.b1995cd6.js
album/assets/[email protected]~swiper-bundle.min.css
album/assets/[email protected]~swiper-bundle.min.js
album/assets/vendor.50b6404e.js
album/carousel.html
album/photos.html
beach.jpg
bobcat.jpg
coconuts.jpg
lake.jpg
sunset.jpg

BUCKET developer-b2-quick-start CONTAINS 14 OBJECTS

Process finished with exit code 0

This concludes our review of the sample application's function list_object_keys() and the logic in the conditional block in the sample application's main() function that both calls it and then uses print() statements to output the values returned.

You can explore the remainder of this code on your own. Or you can go to the next lesson in this series.