Why your fine-tuning job takes longer than the FLOPs say

Practical AI: Tools, Models & Frameworksfine-tuning

What changed

Researchers from Dharma-AI analyzed 272 production supervised fine-tuning jobs (7,382 GPU-hours on H200s) and built a machine learning predictor that estimates end-to-end fine-tuning duration with 1.71-hour mean absolute error, versus 12.41 hours for standard FLOP-based estimates. A CatBoost regressor trained on job configuration and overhead-aware features (evaluation volume, checkpoint I/O, per-GPU load) achieves R² = 0.98, compared to R² = 0.31 for analytical FLOP math.

Why it matters

Fine-tuning jobs on shared GPU clusters routinely run 12+ hours off their FLOP-based estimates, causing idle GPU time or cascading delays. Accurate wall-clock predictions before dispatch enable better cluster scheduling decisions—which jobs run together, when GPUs free up, and whether deadlines are met. The model shows evaluation cadence and checkpoint policy significantly impact runtime, suggesting configuration choices often made by habit are primary levers for duration control.

How it compares

Unlike public fine-tuning APIs (GPT-4o, Gemma) that abstract away cluster scheduling, this addresses the operational problem for teams running private shared infrastructure where GPU allocation and job queueing directly impact throughput and cost. The focus is not on making fine-tuning faster, but on predicting wall-clock time accurately enough for production scheduling—a gap the existing fine-tuning service offerings do not publicly address.

Sources