MMPose Video Keypoint Extraction
Extract 2D/3D skeleton keypoints from video frames with a local MMPose engine and store per-frame keypoint JSON, skeleton-overlay images, and a keypoints_index.jsonl dataset manifest on Backblaze B2 — a Next.js + FastAPI sample for building pose datasets that training and analytics pipelines read straight from object storage.
Built for: Sports-science teams and fitness-app developers who need pose data at dataset scale to train pose classifiers, rep-counting models, and biomechanical pipelines.
What it's built with
Each piece of the stack, and the job it does in this project.
- Next.js 16 + React 19
- Web UI: dashboard, ingest, runs, library, and full-bucket file browser
- Tailwind CSS v4 + shadcn/ui
- Design tokens and reusable interface primitives
- TanStack Query + Recharts
- Client-side data fetching and dashboard charts
- FastAPI + Pydantic v2
- Typed Python API with a checked OpenAPI contract
- MMPose / MMDetection / MMCV
- On-device 2D top-down and 3D lifting pose engine (opt-in install)
- torch
- Model runtime, CPU by default with CUDA auto-detect
- boto3 + Backblaze B2
- S3-compatible object storage for frames, keypoints, overlays, and the manifest
- pnpm workspaces
- TypeScript and Python monorepo workflow
Core capabilities
What the project does out of the box, before you write any code of your own.
On-device MMPose engine
A real 2D top-down and 3D lifting pose engine runs per frame and writes back keypoint JSON and skeleton-overlay images. It runs on CPU by default and auto-detects CUDA, natively on macOS arm64 with no container.
Extraction runs as B2 manifests
The Extraction Run entity has a full lifecycle — create, read, edit, delete, and execute — persisted as a Backblaze B2 manifest with no database to operate.
The manifest is the dataset index
Every source frame maps to its derived keypoint JSON and overlay in a keypoints_index.jsonl manifest, so a training job streams the dataset straight from B2.
Full-stack UI over object storage
A Next.js dashboard, ingest, runs, run detail, session library, and a full-bucket file browser cover the whole ingest-to-dataset workflow.
Agent-first, contract-checked backend
A layered FastAPI service with structural tests, a checked OpenAPI contract, and agent-optimized docs so a coding agent can read the repo and extend it safely.
A closer look
More screens from the running project.
Choose it for the right job
MMPose Video Keypoint Extraction is a head start for a specific shape of project, not a supported, general-purpose product.
Use it when
- You need to turn a video or frame library into a pose-keypoint dataset on B2 — per-frame keypoint JSON, skeleton-overlay images, and a JSONL manifest.
- You want to run pose estimation on local open-source models, with your B2 credentials as the only keys required.
- You want a full-stack Next.js + FastAPI scaffold with the extraction-run lifecycle already wired to object storage.
- You plan to stream the dataset into a training or analytics pipeline directly from B2 over the S3-compatible API.
Choose another path when
- You need a hosted SaaS with managed hosting, accounts, authentication, tenant isolation, or an SLA.
- You need real-time streaming pose tracking or a clinical-grade biomechanics product.
- You cannot run a machine with CPU or GPU compute for the engine — it does not run on Vercel serverless, so extractions run on a box with the engine installed.
- You only need generic file uploads and object-storage workflows without pose estimation — the Vibe Coding Starter Kit is a simpler starting point.
Project status and support
Report defects through the repository's GitHub Issues; this is an unsupported Backblaze sample provided as-is with no SLA.