Accelerated Inference with Optimum and Transformers Pipelines

What changed

Inference has landed in Optimum with support for Hugging Face Transformers pipelines, including text-generation using ONNX Runtime. The adoption of BERT and Transformers continues to grow. Transformer-based models are now not only achieving state-of-the-art performance in Natural Language Processing but also for Computer Vision, Speech, and Time-Series.

Why it matters

A concrete addition to Practical AI: Tools, Models & Frameworks: it changes what's available to builders today rather than being general commentary.

How it compares

Related prior coverage to compare against:

• Introducing Optimum: The Optimization Toolkit for Transformers at Scale
• Few-shot learning in practice: GPT-Neo and the 🤗 Accelerated Inference API
• Blazing Fast SetFit Inference with 🤗 Optimum Intel on Xeon

• Introducing Optimum: The Optimization Toolkit for Transformers at Scale similarity 0.36
• Few-shot learning in practice: GPT-Neo and the 🤗 Accelerated Inference API similarity 0.29
• Blazing Fast SetFit Inference with 🤗 Optimum Intel on Xeon similarity 0.24