Recently, during an event in London, Meta executives Nick Clegg and Yann LeCun announced the imminent release of Llama 3. This unveiling marks the debut of the third and fourth major open-source models to hit the market this month, following xAI’s Grok-1.5V and Mistral’s 8x22B.
Llama 3 boasts impressive pre-training on 15 trillion tokens, a remarkable increase compared to its predecessor, Llama 2. Additionally, the pretraining data now encompasses four times more code. Under the hood, Llama 3 introduces architectural enhancements, including a more efficient tokenizer with a larger vocabulary of 128K tokens.
Here’s a brief overview of Llama 3’s performance:
- In the 8B category, Llama 3 outperforms models like Mistral’s 7B and Google’s Gemma 7B across various benchmarks.
- It excels in several areas, such as MMLU, ARC, DROP, GPQA (primarily science-based questions), HumanEval (code generation), GSM-8K (math problems), MATH (math benchmark), AGIEval (problem-solving), and BIG-Bench Hard (commonsense reasoning).
In the 70B category, Llama 3 remains competitive with top AI models like Google’s Gemini 1.5 Pro, surpassing it in key benchmarks such as MMLU, HumanEval, and GSM-8K. It also outperforms Anthropic’s Claude 3 Sonnet on multiple benchmarks, including MMLU, GPQA, HumanEval, GSM-8K, and MATH.
Llama 3 has Impressive Scores
These impressive scores establish Llama 3 as the new top-performing open-source model, despite some limitations imposed by Meta’s licensing.
Moreover, Llama 3 promises to be more user-friendly, with fewer non-responses and higher accuracy for trivia questions, historical facts, and STEM-related queries. It is set to become widely available across major platforms, including cloud services and API providers.
Meta’s approach to generative AI sets it apart as somewhat of a rebel in the industry. Yann LeCun, Meta’s Chief AI Scientist, has been vocal about his views on AI’s direction, which diverge from those of Meta’s competitors. Meanwhile, Nick Clegg, Meta’s head of global affairs, has faced criticism for his stance on Meta’s AI products.
Despite Meta’s optimism about the quality of its models, concerns persist about the potential societal impacts of unchecked model growth. The release of Llama 3 coincides with controversies surrounding Meta’s AI Facebook agents, underscoring the need for responsible AI development.
Llama 3’s launch, along with models like Grok-1.5 and Mistral’s offerings, represents a shift towards greater empowerment of open-source communities in the generative AI market. As the landscape evolves, the focus now turns to players like Microsoft-OpenAI, poised to make their next move in this dynamic game of Gen-AI chess.
