Imagine a live tutor within Windows that could walk you through an application, showing you what to do. That’s what Microsoft has in mind for Copilot Vision for Windows.
Microsoft is celebrating its 50th anniversary by showing off Copilot Vision for mobile (Android and iOS) as well as Copilot for Windows. Copilot Vision for mobile is available today. Copilot Vision for Windows is scheduled to debut for members of Microsoft’s Windows Insider blog next week and deploy more broadly later on within the existing Windows app.
Microsoft showed off Copilot Vision for Windows in a recorded demonstration released Friday. In the video, the user asked Copilot Vision, who was “following” the user as she worked in Adobe Photoshop, how to adjust the saturation. Copilot responded in the affirmative, then told the user what to do, highlighting the proper control to click. Copilot guided the user step by step — never actually performing the task, but visually indicating, via highlights, exactly what the user needed to do.
Copilot Vision for mobile sounds like Google’s Project Astra: using your phone’s camera to “see” the world, Copilot can used to answer questions about the world around you. Microsoft suggested that Copilot Vision could be used to assess the health of your plants, or to provide decorating tips. But it will be up to the user to use Copilot Vision, ask the right questions, and for Copilot to provide the right answers.
Microsoft
The new feature will undoubtedly be controversial. When Microsoft debuted its Recall feature last year, a portion of the potential userbase protested, loudly, about how poorly Microsoft implemented Recall. Microsoft then went back to secure Recall and the angry cries quieted down.
Recall, however, was a “camera” that always snapped photos of what you saw, periodically, and stored them in what was originally a largely unsecured database. Copilot Vision, presumably, will only turn on when you enable it, and will only see what you’re looking at. Microsoft suggests that the Alt+SPACE shortcut could be held down for about two seconds to enable this feature. (Microsoft has also begun testing a long-press of the Copilot key to enable the same thing.)
Microsoft
If Copilot Vision can deliver on its promises — and yes, that could be a big if — it certainly could be useful. It sounds like we’ll find out for ourselves in a few weeks.
