Large files can range from 5MB all the way up to 10TB, well beyond the 5GB limit on normal files.
The 5GB limit on normal files is there primarily because that's how much you can reliably upload with a single HTTP request in a reasonable amount of time. Large files are created by assembling parts, where each part can either be uploaded or copied from existing files in any bucket belonging to the same account as the large file. And just to be clear, that means a large file can be assembled from a mix of uploaded and copied parts.
Parts of a large file can be uploaded and copied in parallel,
which can significantly reduce the time it takes to upload terabytes of
data. Each part can be anywhere from 5MB to 5GB in size, and
you can pick the size that is most convenient for your application.
For best upload performance, we recommend using the
Creating Large Files
The steps to create a large file are:
b2_get_upload_part_urlfor each thread uploading,
b2_copy_partfor each part of the file, and
The first step is to tell B2 that you're going to create a large file
and providing the file name, content type, and custom file info. The call
will return a file ID for the large file, which you must have when
uploading the parts of the file.
Then, you need to decide where your parts are coming from. Each part can either be uploaded, or copied from an existing file in any bucket belonging to the same account as the large file. Note that large files can be assembled from a mix of uploaded and copied parts.
You need to determine the size of each part you upload and copy. For example, if you're uploading a 100GB file, you could make each part 1GB, and perform 100 uploads, 1GB at a time. Note that parts need not all be the same size. The maximum part size is 5GB, and the minimum part size is 5MB, except for the last part in a file, which has a minimum size of one byte. We recommend a part size of 100MB, which strikes a good balance between upload throughput and the ability to upload parts in parallel.
Rather than hard coding the part size, it is a good idea to use the
The parts are numbered starting at 1, up to the number of parts needed, with a maximum of 10,000 parts for one large file.
to get the target for uploading parts. Each thread doing uploading
should get its own URL.
Upload a part using
provide the file ID of the large file, the part number, and the
data in the part.
Copy a part using
provide the source file id, the large file id, the part number,
and optionally the range of bytes to copy over from the source file.
Finally, once all of the parts are uploaded, you can call
to transform the parts into a single B2 file. Once this is done,
it looks just like any other file. You can download it,
and it will show up when you list the files in a bucket.
Managing Large Files In Progress
Any number of large files can be in progress at once. You can
to get a list of them.
For any one unfinished large file, you can use
b2_list_parts to get a list
of the parts that have been uploaded so far.
If you have started a large file, but don't want to finish, you
to delete all of the parts that have been uploaded so far.
Usage Charges for Large Files
In most ways, large files are treated the same as small files. The costs for the API calls are the same.
You will be charged for storage for parts that have been uploaded or copied.
Usage is counted from the time the part is stored. When you call
the parts are combined into one big file, but the number of bytes stored
remains the same, so it doesn't affect the charge for the storage.
Large files do not require a SHA1 checksum on the entire file but Backblaze recommends one.
If the caller knows the SHA1 of the entire large file being created,
Backblaze recommends specifying the SHA1 in the fileInfo during the call to
Inside the fileInfo specify one of the keys as
and for the value use a 40 byte hex string representing the SHA1.
For each part of the large file, a SHA1 is required. When uploading a part, you must provide a SHA1 checksum, which is used to validate the data sent, and which is stored internally with that part.
When you download a large file, or a range of a large file, there is no checksum
for the entire file, so the string
none is returned in the
Accessing Large Files
Once a large file is created, you can do anything you can do with a normal file:
b2_delete_file_version- deletes one version of one file
b2_download_file_by_id- downloads a specific version of a file
b2_download_file_by_name- downloads the most recent version of a file
b2_get_file_info- returns information about a file
b2_hide_file- hides a file, without deleting its data
b2_list_file_names- lists the file names in a bucket
b2_list_file_versions- lists all of the file versions in a bucket
When downloading large files, the
Range header can be
especially useful. It lets you download just part of the file. For
Large files can range in size from 5MB to 10TB.
Each large file must consist of at least 2 parts, and all of the parts except the last one must be at least 5MB in size. The last part must contain at least one byte.