Minions: where local and cloud LLMs meet

What changed

Small LLMs solve these tasks in parallel - Achieves 5.7x reduction in remote costs, while maintaining 97.9% of cloud model performance Get started Clone the repository: git clone https://github.com/HazyResearch/minions.git cd minions Optionally, create a virtual environment with your favorite package manager (e.g. conda, venv, uv, etc.): python3 -m venv .venv source .venv/bin/activate Next, install the Python package and dependencies: pip install -e . If you don’t have it yet, install Ollama and Meta’s Llama 3.2 model: ollama pull llama3.2 Lastly, create an OpenAI API Key for the cloud model.

Why it matters

A concrete addition to Practical AI: Tools, Models & Frameworks: it changes what's available to builders today rather than being general commentary.

How it compares

Related prior coverage to compare against:

• Llama 3.2 goes small and multimodal
• Fine-tuning now available for GPT-4o
• Ollama's new engine for multimodal models

• Llama 3.2 goes small and multimodal similarity 0.15
• Fine-tuning now available for GPT-4o similarity 0.10
• Ollama's new engine for multimodal models similarity 0.10