JuiceFS Integration with Backblaze B2

This guide shows how to deploy JuiceFS with Backblaze B2 Cloud Storage as the object storage backend for AI/ML workloads.

Prerequisites

• A Backblaze B2 account

• A B2 bucket with Lifecycle Rules

  JuiceFS updates object chunks frequently. Without Lifecycle Rules, hidden versions can accumulate.

  To reduce storage overhead, configure:

  • Days from hide to delete: 1

    This keeps the latest version and deletes hidden versions one day after they are hidden.

• An application key with access to the bucket

• A Redis instance reachable from every machine or pod that mounts the filesystem

• A Kubernetes cluster (for CSI deployment) or a Linux host

• kubectl configured (for Kubernetes)

Install JuiceFS

1. Run the following command:

   curl -sSL https://d.juicefs.com/install | sh

2. Verify installation:

   juicefs version

Configure JuiceFS

1. Set environment variables:

   export ACCESS_KEY=<your-key-id>
   export SECRET_KEY=<your-application-key>

2. Create the filesystem:  

   juicefs format \
     --storage s3 \
     --bucket https://s3.<region>.backblazeb2.com/<bucket-name> \
     redis://<redis-host>:6379/1 \
     my-jfs

Mount the Filesystem

1. Mount the filesystem:

   juicefs mount -d redis://<redis-host>:6379/1 /mnt/jfs

   The -d flag runs the mount in the background; without it the command stays in the foreground. On macOS, install macFUSE first.  

2. Validate the mount:  

   echo "hello world" > /mnt/jfs/test.txt
   cat /mnt/jfs/test.txt

Deploy in Kubernetes (CSI)

Follow these steps to install the JuiceFS CSI driver and configure Backblaze B2-backed persistent storage for your Kubernetes workloads.

Install the CSI Driver

Install the JuiceFS CSI driver and create the namespace first.

helm repo add juicefs https://juicedata.github.io/charts/
helm install juicefs-csi-driver juicefs/juicefs-csi-driver \
  --namespace kube-system
kubectl create namespace juicefs-testing

Replace the ${...} placeholders with your values (for example with envsubst), then apply the manifests.

Create a Secret

Create a Kubernetes Secret to store the JuiceFS metadata service and Backblaze B2 connection details.

apiVersion: v1
kind: Secret
metadata:
  name: jfs-b2-secret
  namespace: juicefs-testing
stringData:
  name: "b2-ai-storage"
  metaurl: "redis://<redis-host>:6379/1"
  storage: "s3"
  bucket: "https://<bucket-name>.s3.<region>.backblazeb2.com"
  access-key: "${B2_KEY_ID}"
  secret-key: "${B2_APPLICATION_KEY}"

Create a StorageClass

Create a StorageClass that uses the JuiceFS CSI driver and defines the cache settings for your workloads.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: juicefs-b2-ai-optimized
provisioner: csi.juicefs.com
parameters:
  csi.storage.k8s.io/provisioner-secret-name: jfs-b2-secret
  csi.storage.k8s.io/provisioner-secret-namespace: juicefs-testing
  csi.storage.k8s.io/node-publish-secret-name: jfs-b2-secret
  csi.storage.k8s.io/node-publish-secret-namespace: juicefs-testing
mountOptions:
  - cache-size=51200
  - cache-dir=/data/juicefs-cache
  - attr-cache=300
  - entry-cache=300

The mount options increase the cache size (51200 MiB) and metadata cache (300s) to improve performance for AI/ML workloads.

Run Performance Tests

Run these fio commands from the mounted directory to simulate AI/ML access patterns. The test files are large, so reduce the --size value for a quick test.

Model Loading (Sequential Read)

Test sequential read performance to simulate loading a large model from the mounted file system.

fio --name=ai-model-load \
  --directory=/mnt/jfs \
  --rw=read \
  --bs=1M \
  --iodepth=32 \
  --numjobs=1 \
  --direct=1 \
  --ioengine=libaio \
  --size=10G

Checkpointing (Sequential Write)

Test sequential write performance to simulate saving a model checkpoint.

fio --name=checkpoint-write \
  --directory=/mnt/jfs \
  --rw=write \
  --bs=1M \
  --iodepth=32 \
  --numjobs=1 \
  --direct=1 \
  --ioengine=libaio \
  --size=10G

Training Data Access (Random Read)

Test random read performance to simulate accessing training data during a workload.

fio --name=training-data-read \
  --directory=/mnt/jfs \
  --rw=randread \
  --bs=1M \
  --iodepth=32 \
  --numjobs=1 \
  --direct=1 \
  --ioengine=libaio \
  --size=10G

The --ioengine=libaio option is available only on Linux. Omit this option on macOS.

Additional Resources

• JuiceFS Documentation

• Backblaze B2 documentation

• Backblaze B2 S3-Compatible API documentation

• Backblaze Labs