LLaMA, is Meta preparing for its version of ChatGPT?

It was to be expected that Meta would take the success of ChatGPT as an opportunity. “Today we release a new next-generation artificial language model called LLaMA designed to help researchers advance their work – writes Mark Zuckerberg from his Facebook account -. The LLMs have shown great promise in generating text, having conversations, summarizing written material, and more complicated tasks such as solving math theorems or predicting protein structures. Meta is committed to this open research model and we will make our new model available to the AI ​​research community”.

Until now, LLMs have always been in demand an extremely powerful computing infrastructure for training and execution, making them inaccessible to most researchers. With LLaMA, Meta states that it is democratizing access to LLMs, which are seen as one of the most important and beneficial forms of AI. The most famous example of an LLM is GPT-3 of OpenAI, the model behind ChatGPT, which has recently gained enormous success precisely because of its extraordinary ability to answer (almost) any type of question in a human way. But these are extremely complex models, therefore Meta has decided to focus on designing smaller modelswhich require less computing power to test new approaches, validate the work of others, and explore new use cases.

By making a smaller LLM available to the research community, Meta hopes researchers will be able to better understand how and why they work. Furthermore, LLaMA also seems to have another advantage: it is trained on multiple tokens, making it easier to retrain and tune for specific use cases. The 13 billion parameter model, for example, was trained on 1 trillion tokens. In contrast, GPT-3 was trained on just 300 billion tokens. This makes LLaMA much more versatile, capable of being applied to far more use cases than a finely tuned model like GPT-3, designed for more specific tasks. What does all this mean? What Meta is preparing for its own version of ChatGPT?