According to Intel, Microsoft wants to run parts of Copilot locally on your PC soon. Only: the minimum requirements for this are higher than what Intel can currently offer.
Microsoft wants to run its AI Copilot locally soon. Intel confirmed this to Tom’s Hardware at the AI Summit. Microsoft Copilot is a generative AI assistant based on the same GPT model that powers OpenAI’s ChatGPT. The Copilot can help you with all kinds of daily tasks, such as searching emails for action items or preparing PowerPoints. However, at the moment all queries to Copilot must pass through the cloud, because only there is sufficient computing power for inference.
Local inference
That takes time, network capacity and computing power in the data center. In addition, privacy-sensitive data may have to go back and forth and customers do not always want that. The solution is to run Copilot locally, at least for slightly lighter tasks. This is possible, as long as the local hardware has sufficient computing power to efficiently process such inference tasks.
Intel even communicates a figure: Microsoft wants 40 TOPS of NPU computing power. The same figure was previously released by Trendforce and we also saw it on slides from HP. Striking: Microsoft itself prefers to remain vague about it at the moment and does not want to confirm or deny the 40 TOPS to the editors. Fortunately, Intel does that instead of Microsoft.
Future-oriented minimum
40 TOPS (Tera Operations per Second) is a measure of an NPU’s AI horsepower. At the moment, no computer with only a processor on board has sufficient TOPS. The NPUs on board the processors from Intel and AMD have a maximum of ten and sixteen TOPS. In other words, Copilot cannot run locally on the current generation of ‘AI PCs’ according to Microsoft’s figures, which are now widely confirmed by almost everyone except Microsoft.
Intel is confident that it will reach the lower bound for Copilot performance with the next generation of chips this year. AMD is also undoubtedly aware of the requirements and we would be surprised if new Ryzen chips do not meet the minimum limit.
Microsoft’s plans show where AI is evolving. The Copilot must become ubiquitous using local hardware. In fact, Intel is partly making a sales pitch for Nvidia and AMD: today only computers with a discrete GPU on board have the necessary TOPS.
