Microsoft Copilot

2024: Recall takes thousands of screenshots of a user every hour and stores recognized text on Microsoft servers

By June 2024, it became clear that Microsoft had decided to take user control to a whole new level. The capabilities of AI in conjunction in cloud services now allow you to do this.

Constant monitoring of user activity is planned to be carried out through the new Recall function, the implementation of which was announced at the Surface event organized by Microsoft.

Recall takes thousands of screenshots every hour. Snapshots are analyzed using AI, which allows you to configure the algorithm for finding text and images. It is this functionality that Microsoft representatives call "photographic memory" for your PC.

Despite criticism from cybersecurity experts, the launch of the new functionality is scheduled in the coming days in parallel with the start of sales of Copilot + PCs.

Recall is one of several AI features that will appear exclusively on Copilot + computers. These devices, which include Microsoft's new Surface Laptop 7, require a dedicated neural processing unit (NPU) to perform AI tasks.

Representatives of Samsung, Dell, Asus and other computer manufacturers have already promised to release Copilot + computers.

The British data protection body has already confirmed that it is investigating Microsoft, and the head of SpaceX, billionaire Elon Musk, has warned millions of subscribers on site X to abandon this feature, comparing what is happening to an episode of the popular series Black Mirror.

Cybersecurity expert and former Microsoft employee Kevin Beaumont discovered that Recall stores data in a database in plain text, without encryption. A hacker can easily get data about everything you did on your computer.

Screenshots are automatically recognized by the Azure AI AI running on your device and written to the SQLite database in the user's folder. This database file stores a record of everything you've ever viewed on your computer in plain text.

The analysis is based on the OCR (Optical Character Recognition) algorithm - optical character recognition. This is an algorithm for digitizing images of typed, handwritten or printed text. Microsoft uses AI to decrypt text from web pages, MS Word documents, PDFs, handwritten notes and everything else that appears on your computer's screen so it can all be found instantly.

Beaumont called the feature a "disaster," warning that "stealing anything you've ever typed or viewed on your Windows PC is now possible with a couple of lines of code."

2023: Copilot included in Windows 11

Microsoft released a new pre-build Windows of 11 for Windows Insider members on the Dev channel. Assembly 22494 contains a number of additional features and improvements, including support for Copilot AI. This became known on June 29, 2023.

AI Copilot is positioned as an intelligent assistant that can help users with a variety of tasks such as finding information, creating documents, editing photos and videos, learning new skills, and more. AI Copilot is integrated into the taskbar and can be activated by voice or keyboard. Users can ask him questions, ask for advice or give commands in natural language.