Source Code Bundles
Large files can range from 100MB all the way up to 10TB, well beyond the 5GB limit on normal files.
The 5GB limit on normal files is there primarily because that's how much you can reliably upload with a single HTTP request in a reasonable amount of time. Large files let you upload files in multiple parts.
Parts of a large file can be uploaded in parallel, which can
significantly reduce the time it takes to upload terabytes of
data. Each part can be anywhere from 5MB to 5GB in size, and
you can pick the size that is most convenient for your application.
For best upload performance, we recommend using the
Uploading Large Files
The steps to upload a large file are:
The first step is to tell B2 that you're going to upload a large file
Then, you need to decide what size each upload for each part. For example, if you're uploading a 100GB file, you could make each part 1GB, and upload 1GB a time. The maximum part size is 5GB, and the minimum part size is 5MB, except for the last part in a file, which has a minimum size of one byte. We recommend a part size of 100MB, which strikes a good balance between upload throughput and the ability to upload parts in parallel.
Rather than hard coding the part size, it is a good idea to use the
The parts are numbered starting at 1, up to the number of parts needed, with a maximum of 10,000 parts for one large file.
Upload each part using
Finally, once all of the parts are uploaded, you can call
Any number of large file uploads can be in progress at once. You can
For any one upload, you can use
If you have started uploading a large file, but don't want to finish, you
Usage Charges for Large Files
In most ways, large files are treated the same as small files. The costs for the API calls are the same.
You will be charged for storage for parts that have been uploaded. Usage
is counted from the time the upload is done. When you call
Large files do not require a SHA1 checksum on the entire file but Backblaze recommends one.
If the caller knows the SHA1 of the entire large file being uploaded,
Backblaze recommends specifying the SHA1 in the fileInfo during the call to
For each part of the large file, a SHA1 is required. You must provide a SHA1 checksum when you upload it, which is used to validate the data sent, and which is stored internally with that part.
When you download a large file, or a range of a large file, there is no checksum
for the entire file, so the string
Accessing Large Files
Once a file is uploaded, you can do anything you can do with a normal file:
Large files can range in size from 5MB to 10TB.
Each large file must consist of at least 2 parts, and all of the parts except the last one must be at least 5MB in size. The last part must contain at least one byte.