Why we’re switching to Hugging Face Inference Endpoints, and maybe you should too
What changed
- Deploy (almost) any model on Hugging Face Hub - To any cloud (AWS, and Azure, GCP on the way) - On a range of instance types (including GPU) - We’re switching some of our Machine Learning (ML) models that do inference on a CPU to this new service. This blog is about why, and why you might also want to consider it. The models that we have switched over to Inference Endpoints were previously managed internally and were running on AWS Elastic Container Service (ECS) backed by AWS Fargate.
Why it matters
A concrete addition to Practical AI: Tools, Models & Frameworks: it changes what's available to builders today rather than being general commentary.
How it compares
Related prior coverage to compare against:
- Deploy LLMs with Hugging Face Inference Endpoints
- Deploy Embedding Models with Hugging Face Inference Endpoints
- Getting Started with Hugging Face Inference Endpoints
Sources
- Why we’re switching to Hugging Face Inference Endpoints, and maybe you should too (huggingface-blog)primary