Microsoft processed 3.8 billion parameters in Phi-3 Mini. The LLM is considerably smaller than general knowledge competitors GPT-4 and Llama 3, but focuses on vertical applications.
The latest version of Microsoft’s smallest LLM (Large Language Model), Phi-3, is now available on Azure, Hugging Face, and Ollama. The model is available in three flavors:
- Phi-3 Mini with 3.8 billion parameters (available now)
- Phi-3 Small with 7 billion parameters (coming soon)
- Phi-3 Medium with 14 billion parameters (available soon)
At the end of last year, Microsoft launched its predecessor, Phi-2, the smallest model of which contained 2.7 billion parameters. The latest version is said to perform better and deliver answers comparable to LLMs that are ten times larger.
The big advantage of small AI models is that they require less energy to run. This reduces energy costs and makes mobile applications possible. Microsoft isn’t the only one working on small LLMs. Google, among others, has two variants: Gemma (2 billion and 7 billion parameters). Anthropic also has a compact version of Claude 3 called Haiku. Last week, Meta launched Llama 3 in two variants: 8 billion and 70 billion parameters.
Microsoft says it trained Phi-3 with a “curriculum” inspired by how children learn from bedtime stories. The focus of Phi-3 is on coding and reasoning. The software giant says that smaller LLMs are a logical choice for many companies. Their internal datasets are more compact and models like Phi-3 are cheaper to run because they require less computing power and energy.
Curious about more information about artificial intelligence? Then be sure to take a look at our AI theme page with video and three interesting, detailed articles.
