StarCoder: A State-of-the-Art LLM for Code
What changed
StarCoder and StarCoderBase are Large Language Models for Code (Code LLMs) trained on permissively licensed data from GitHub, including from 80+ programming languages, Git commits, GitHub issues, and Jupyter notebooks. Similar to LLaMA, we trained a ~15B parameter model for 1 trillion tokens. We fine-tuned StarCoderBase model for 35B Python tokens, resulting in a new model that we call StarCoder.
Why it matters
A concrete addition to Practical AI: Tools, Models & Frameworks: it changes what's available to builders today rather than being general commentary.
How it compares
Related prior coverage to compare against:
- Introducing IDEFICS: An Open Reproduction of State-of-the-art Visual Langage Model
- Llama 3
- Introducing ChatGPT Images 2.0
Sources
- StarCoder: A State-of-the-Art LLM for Code (huggingface-blog)primary