Create Large Files with the Native API
There are four Backblaze B2 Cloud Storage Native API calls that you need to make to create a large file.
b2_get_upload_part_url(for each thread that are are uploading)
b2_copy_part(for each part of the file)
b2_start_large_file and provide the file name, content type, and custom file information. The call returns a file ID for the large file, which you need when you upload the parts of the file.
You can either upload each part of your file, or you can copy them from an existing file in any bucket that belongs to the same account as the large file. Note that you can assemble large files from a mix of uploaded and copied parts.
Determine the size of each part that you upload and copy. For example, if you want to upload a 100 GB file, you can make each part 1 GB, and perform 100 uploads 1 GB at a time. The file parts do not need to be the same size. The maximum part size is 5 GB, and the minimum part size is 5 MB except for the last part in a file which has a minimum size of 1 byte. Backblaze recommends a part size of 100 MB which strikes a good balance between upload throughput and the ability to upload parts in parallel.
Rather than hard coding the part size, you can use the
recommendedPartSize that is returned by the
The parts are numbered starting at one, up to the number of parts that are needed, with a maximum of 10,000 parts for one large file.
b2_get_upload_part_url to get the target for uploading parts. Each thread that uploads should get its own URL.
Upload a part using
b2_upload_part, and provide the file ID of the large file, the part number, and the data in the part.
Copy a part using
b2_copy_part, and provide the source file ID, the large file ID, the part number, and optionally the range of bytes to copy over from the source file.
Finally, after all of the parts are uploaded, you can call
b2_finish_large_file to transform the parts into a single Backblaze B2 file. After this is done, it looks just like any other file. You can download it, and it will show up when you list the files in a bucket.
Manage Large Files in Progress
Any number of large files can be in progress at the same time. You can use
b2_list_unfinished_large_files to get a list of them.
For any one unfinished large file, you can use
b2_list_parts to get a list of the parts that were uploaded so far.
If you started a large file, but do not want to finish, you can use
b2_cancel_large_file to delete all of the parts that were uploaded so far.
Access Large Files
After a large file is created, you can do anything you can do with a normal file:
b2_delete_file_versionThis operation deletes one version of one file.
b2_download_file_by_idThis operation downloads a specific version of a file.
b2_download_file_by_nameThis operation downloads the most recent version of a file.
b2_get_file_infoThis operation returns information about a file.
b2_hide_fileThis operation hides a file without deleting its data.
b2_list_file_namesThis operation lists the file names that are in a bucket.
b2_list_file_versionsThis operation lists all of the file versions that are in a bucket.