Meta recently unveiled its latest artificial intelligence models, Llama 3 8B and 70B, which are set to revolutionize the world of AI. With improved capabilities and new training methods, these models are expected to outperform their predecessors. Excitingly, Meta has announced that their large models will now contain over 400 billion parameters, a significant increase from the 70B model in Llama 2.
Meta is taking a community-first approach with the release of Llama 3, making the new foundation models open source. This move will allow enthusiasts and developers to access the AI models through various platforms such as AWS, Google Cloud, and Microsoft Azure. The integration of Llama 3 with Meta AI on platforms like Facebook Messenger, Instagram, and WhatsApp will also provide users with easy access to the models in supported countries.
In terms of performance, Meta shared benchmark scores for Llama 3, showcasing its superiority over competitors. The pre-trained model of Llama 3 70B outperformed Google’s Gemini 1.0 Pro in various benchmarks, highlighting its conversational AI capabilities. On the other hand, the instruct models of Llama 3 demonstrated better performance in completing specific tasks compared to Gemini 1.5 Pro.
Advanced Architecture
Meta has implemented a decoder-only transformer architecture for the new AI models, with several enhancements to improve efficiency. Llama 3 now utilizes a tokeniser with a vocabulary of 128K tokens, along with grouped query attention (GQA) for better inference. The adoption of GQA ensures that the AI remains focused within its context when responding to queries, enhancing the overall user experience.
One of the key aspects of Llama 3 is its extensive data training, with Meta revealing that the models have been pre-trained with over 15 trillion tokens sourced from publicly available data. This vast amount of data ensures that the AI models are well-equipped to handle a wide range of tasks and queries, making them versatile and reliable.
Meta’s introduction of Llama 3 8B and 70B marks a significant milestone in the field of artificial intelligence. With enhanced capabilities, open-source availability, and superior performance, these models are poised to drive innovation and development in the AI industry. As technology continues to evolve, Meta’s commitment to advancing AI research and development is evident in the groundbreaking features of Llama 3.
Leave a Reply