Meta won’t matter much once Apple shares what it has been working on. Apple has shared some details concerning accessibility features it is working on, dropping some pretty big hints at how it sees augmented everyday reality. Will we see more on this at WWDC 2022, and how will it be applied? Making life accessible, making reality data Two of the upcoming accessibility enhancements seem to suggest Apple’s approach: Door Detection and Live Captions. Here’s what they do: Door Detection: Using the iPhone camera, it will detect a door, navigate a user to that door, inform them if the door is open or closed, tell them how to open the door, and it can understand and read things like door numbers. Live Captions: Your Apple device will listen to any audio content and give you a real-time transcript of that conversation. Both are incredible features, but when you consider them a little, they become quite amazing. I see it this way: once an Apple device can create a real-time transcript of what it hears, why should it be unable to translate that transcript into different languages? What this could mean We know Apple has the technology to do this — we use it each time we translate a web page. That process is super-fast, so why not simply extend that translation to the transcription delivered by your Apple device? This could work two ways, also, with your device speaking the language you can’t, enabling you to join complex conversations across multiple tongues. Door Detection uses technologies Apple has been exploring for some time. You can easily use them yourself — open Photos and search for images of “Lamp Post” and you’ll be able to explore every photo you have that includes a lamp post. Now, I don’t know about you, but if your device can recognize items in photos, it should be able to recognize them elsewhere using the same machine vision intelligence. Vision + Intelligence + Context =? That means that, just as a person who is blind or has low vision can look forward to using Door Detection to find and open a door, it’s reasonable to think they’ll be able to use similar technology to recognize anything else the AI on Apple devices has a name for: “Hey Siri, where are the oranges in the vegetable store?” “They’re three steps to your right, in the box second from the front. They cost $1.” Door Detection tells us this will happen because the technology already exists to enable it. It just needs building out. So, what’s revolutionary about all of this? It means Apple has already assembled a host of building blocks that enable its technologies to recognize and interact with the world around us. Once technology understands that worls, it can help guide our interactions, augmenting our decisions with information we can use. A blind or low-vision person about to buy a $1 orange might be told the same fruit is available for half that price further down the street. Or a field service engineer might find their device has already opened the troubleshooting manual for the hardware they happen to be staring at. [Also read: Apple calls out Meta for hypocrisy] What we have here are two technologies, ostensibly built for accessibility, that also give the company’s devices interactive understanding around sight and sound. That understanding enables the device to provide contextually useful information concerning what is seen and what is heard to the user. This can be in response to direct questions, or, reflecting the work Apple has been doing with Siri Suggestions, driven by the device’s knowledge of the kind of help you usually request. The augmentation of human experience has begun You don’t need to be an enterprise professional to recognize that this opens a range of opportunities for powerful tools and services for consumer users — along with profoundly powerful enterprise applications around machine vision intelligence and Industry 5.0 across multiple sectors. One of the great things about those applications is that, because they are based on accessibility technologies, they also enable those who may not yet be as equally represented as they should be in some fields to take more active part. That’s what I call augmented reality. And that’s what I think we’re about to learn a great deal more about at WWDC 2022. No wonder Apple has begun to leak information about showcasing these technologies to company directors and the design challenges that went into developing the most logical vehicle for such tech, Apple Glass. Step-by-step, the building blocks of this multi-year effort are falling into place more rapidly now. I can already hear the critics getting ready to be wrong again. Please follow me on Twitter, or join me in the AppleHolic’s bar & grill and Apple Discussions groups on MeWe. Related content news analysis Apple Silicon sets scene for a new AI ecosystem With its new iPads, Apple presses home the message that Apple Silicon is built for AI. By Jonny Evans May 08, 2024 12 mins Apple Generative AI iPad news analysis 3+ reasons Apple might want to make its own server chips Apple reportedly has a top secret plan to make AI chips for servers to provide generative AI services. By Jonny Evans May 07, 2024 5 mins Apple CPUs and Processors Generative AI news With its new iPad, Apple's Empire strikes back Apple is preparing to introduce new iPad Pro and iPad Air models as it seeks to regain momentum in the tablet market. By Jonny Evans May 06, 2024 5 mins iPad Apple Tablets news analysis Apple earnings: About that iPhone 'slump' in China Based on information from Thursday's earnings report, it seems that data pointing to an iPhone slump in China were over-baked. By Jonny Evans May 03, 2024 9 mins iMac iPhone Apple Podcasts Videos Resources Events SUBSCRIBE TO OUR NEWSLETTER From our editors straight to your inbox Get started by entering your email address below. Please enter a valid email address Subscribe