Introducing the AMD 5th Gen EPYC™ CPU

What changed

It provides a significant boost in performance, especially with a higher number of core count reaching up to 192 and 384 threads. From Large Language Models (LLMs) to RAG scenarios, Hugging Face users can leverage this new generation of servers to enhance their performance capabilities: - Reduce the target latency of their deployments. Furthermore, we have developed an optimized Dockerfile that will be released soon, along with the benchmarking code.

Why it matters

A concrete addition to Practical AI: Tools, Models & Frameworks: it changes what's available to builders today rather than being general commentary.

How it compares

Related prior coverage to compare against:

• AMD and OpenAI announce strategic partnership to deploy 6 gigawatts of AMD GPUs
• Scaling-up BERT Inference on CPU (Part 1)
• Scaling up BERT-like model Inference on modern CPU - Part 2

• AMD and OpenAI announce strategic partnership to deploy 6 gigawatts of AMD GPUs similarity 0.27
• Scaling-up BERT Inference on CPU (Part 1) similarity 0.22
• Scaling up BERT-like model Inference on modern CPU - Part 2 similarity 0.18