Design a Network for High-Throughput Access to Backblaze B2

Backblaze B2 Cloud Storage supports high-throughput data movement through regional, HTTPS-based endpoints. For standard B2 Cloud Storage, performance depends on the path between your systems and the selected Backblaze region, the size and number of objects, and the way your transfer software uses multipart operations and concurrency. For workloads that require designated networking capacity or private networking options, contact Backblaze Sales to evaluate B2 Overdrive.

This guide explains how to design and test a network path for bulk uploads, large restores, downloads, and ongoing application traffic. It uses the S3-Compatible API as the primary path and includes the Backblaze B2 Command-Line Tool as an alternative.

Prerequisites

Before you begin, ensure that you have the following items:

• A Backblaze account with Backblaze B2 enabled

• A private Backblaze B2 bucket and its S3 endpoint

• An Application Key with access to the bucket and the operations in this guide

• A test host with representative CPU, memory, storage, and network capacity. See Cloud Storage Performance.  

• Representative test objects, including the object sizes used by your production workload. For information about multipart files, see     Cloud Storage Large Files.  

• Python 3 and Boto3 for the S3-Compatible API example, or the Backblaze B2 Command-Line Tool for the CLI example.  

• Permission to review firewall, proxy, NAT, and bandwidth settings along the transfer path. If your organization uses a firewall or network allowlist, see Backblaze IP Addresses.  

Choose a Network Pattern

Select the simplest pattern that meets your performance, security, and cost requirements. A lower egress charge does not, by itself, mean that a connection is private.

Use the Endpoint for your Backblaze Region

The endpoint is the URL that an S3-compatible application, SDK, or command-line tool uses to connect to Backblaze B2. Each endpoint identifies the Backblaze region where your account and bucket are located.

The S3-Compatible API uses the following endpoint format:

https://s3.<region>.backblazeb2.com

For example:

https://s3.us-west-004.backblazeb2.com

Copy the S3 endpoint from your bucket details in the Backblaze web console. Enter this value in the endpoint setting for your application, SDK, or integration. The setting might be labeled Endpoint, Endpoint URL, S3 Endpoint, or Server URL.

The endpoint must match the region of your Backblaze account. Do not enter an AWS endpoint, an AWS region name, or an endpoint from another Backblaze account.

Measure the Full Transfer Path

Test from the systems that will run the production workload. A speed test to an unrelated destination does not measure the route to your B2 endpoint.

1. Confirm that DNS resolves the correct endpoint and that outbound HTTPS connections are allowed.

2. Measure round-trip latency and packet loss from each planned transfer location.

3. Run a single-stream upload and download with a representative object.

4. Repeat the test with multipart transfers and gradually increase concurrency.

5. Record throughput, duration, retry count, errors, CPU use, memory use, and local disk performance.

6. Repeat the test at different times and report median and high-percentile results, not only the fastest run.

If performance stops improving as concurrency increases, identify the saturated component before adding more workers. The limit might be the client host, local storage, NAT gateway, firewall, proxy, internet service provider, remote route, or API request rate.

Use Multipart Transfers and Controlled Concurrency

A single connection often cannot fill a high-capacity network path. Multipart transfers divide a large object into parts that a client can transfer concurrently. Concurrency can also improve aggregate throughput when you transfer many independent objects.

The following Python example uses Boto3 with the Backblaze B2 S3-Compatible API. The values are starting points for testing, not universal recommendations. Increase or decrease them based on object size, available memory, network capacity, and observed error rates.

import os

import boto3
from boto3.s3.transfer import TransferConfig
from botocore.config import Config

endpoint = os.environ["B2_S3_ENDPOINT"]
key_id = os.environ["B2_KEY_ID"]
application_key = os.environ["B2_APPLICATION_KEY"]
bucket = os.environ["B2_BUCKET"]

s3 = boto3.client(
    "s3",
    endpoint_url=endpoint,
    aws_access_key_id=key_id,
    aws_secret_access_key=application_key,
    config=Config(
        signature_version="s3v4",
        retries={"max_attempts": 10, "mode": "standard"},
        max_pool_connections=32,
    ),
)

transfer_config = TransferConfig(
    multipart_threshold=64 * 1024 * 1024,
    multipart_chunksize=64 * 1024 * 1024,
    max_concurrency=16,
    use_threads=True,
)

s3.upload_file(
    Filename="large-dataset.bin",
    Bucket=bucket,
    Key="datasets/large-dataset.bin",
    Config=transfer_config,
)

Store credentials in a secrets-management system and inject them at runtime. Do not place application keys directly in source code.

For downloads, use the same client and transfer configuration:

s3.download_file(
    Bucket=bucket,
    Key="datasets/large-dataset.bin",
    Filename="restored-dataset.bin",
    Config=transfer_config,
)

When you use the B2 Native API for large files, each upload thread should obtain its own upload-part URL. Follow the part-size guidance returned by the authorization response and stay within the documented part-count limits.

Tune Transfers with the B2 Command-Line Tool

The b2 sync command supports a configurable thread count. Start with a moderate value, measure the result, and increase it gradually while monitoring the host and error rate.

b2 sync --threads 16 /data/export b2://example-bucket/export

To test downloads, reverse the source and destination:

b2 sync --threads 16 b2://example-bucket/export /data/restore

Do not assume that a larger thread count always produces higher throughput. Excess concurrency can increase retries, memory use, local disk contention, and request volume.

Use a CDN or Partner for Repeated Downloads

A CDN is useful when many users repeatedly request the same objects. Cached objects can be delivered from locations closer to users, reducing latency and requests to the B2 origin.

Backblaze offers free egress to or through specific CDN and compute partners. The current pricing page lists eligible partners, including Cloudflare, Fastly, bunny.net, CacheFly, CoreWeave, Equinix Metal, Vultr, and phoenixNAP. Confirm the current partner list and service-specific conditions before you design a production architecture around this benefit.

Partner egress benefits and private networking are separate concepts. A CDN or compute partner might reduce Backblaze egress charges without giving your organization a dedicated private circuit to the standard B2 endpoint.

Evaluate B2 Overdrive for Specialized Requirements

Evaluate B2 Overdrive when the standard B2 service does not meet a documented throughput or networking requirement. B2 Overdrive provides designated storage and networking resources, supports throughput tiers from 100 Gbps up to 1 Tbps, and offers private networking options.

Contact Backblaze Sales to validate the following details before you commit to the design:

• Available regions and facility locations

• Supported private networking providers and handoff types

• Minimum capacity, storage, and contract requirements

• Provisioning lead time

• Redundancy and failover options

• Target throughput and workload profile

Do not describe a standard B2 S3 endpoint as an AWS Direct Connect, Azure ExpressRoute, or Google Cloud Interconnect endpoint. If your architecture requires a private handoff, confirm the supported B2 Overdrive design with Backblaze before implementation.

Estimate Transfer Duration

Use the following formula for a theoretical minimum:

transfer time in seconds = (data size in bytes × 8) ÷ throughput in bits per second

For example, transferring 10 TB over a fully utilized 10 Gbps connection has a theoretical minimum of approximately 2 hours and 13 minutes. Real transfers take longer because of protocol overhead, latency, retries, multipart coordination, client limits, local storage performance, and competing traffic.

Base production estimates on measured effective throughput from a representative test. Include enough time for retries and validation.

Plan for Request Limits and Retries

New Backblaze accounts begin with a default limit of 500 upload and download requests per second. High concurrency, many small objects, or a large worker fleet can reach this limit even when the network still has unused capacity.

Implement retries with exponential backoff and jitter for retryable responses. If a representative workload requires a higher request rate, collect measurements and contact Backblaze to request a review. Do not retry authentication, authorization, or invalid-request errors without correcting the request.

Troubleshooting

Use the following checks to identify the component that limits throughput before you change the architecture.

Frequently Asked Questions

Does Backblaze B2 support private peering?

Standard B2 Cloud Storage is accessed through regional public HTTPS endpoints. B2 Overdrive offers private networking options for specialized, throughput-intensive deployments. Contact Backblaze Sales to confirm availability and the supported topology for your workload.

How can I reduce egress costs?

Review the current B2 pricing terms and eligible CDN and compute partners. Backblaze offers free egress to or through listed partners, and B2 Overdrive includes unlimited free egress. A partner egress benefit does not necessarily provide a private connection.

How can I improve throughput?

Use the correct regional endpoint, place transfer workers near the B2 region when practical, use multipart transfers for large objects, run independent transfers concurrently, and verify that local storage and network devices can sustain the target rate. Increase concurrency gradually and use measured results to choose the final settings.

Additional Resources

• Cloud Storage Performance

• Backblaze Cloud Storage Data Regions

• Call the S3-Compatible API

• Cloud Storage Large Files

• Rate Limits

• Use the B2 Sync Command with the CLI

• Cloudflare Integrations

• B2 Cloud Storage Pricing

• B2 Overdrive