Apple Watch's Audio Intelligence, Conversation Summaries, and Real-Time Rewind Enter Beta Testing

As a major preview feature first unveiled at its September hardware event, Apple has now officially moved its Audio Intelligence technology for Apple Watch into beta testing. The first wave is open to developers on iOS 27.3, and it is primarily available on two flagship devices: Apple Watch Series 12 and Apple Watch Ultra 4.

Four Core Features: Covering Sound Recognition, Song Recognition, and Smart Summaries

This launch of Audio Intelligence includes four AI features that users can choose to enable entirely on their own. It is designed to comprehensively expand the practical value of the smartwatch in sound perception and everyday life scenarios:

Sound Recognition: This feature is designed specifically for deaf or hard-of-hearing users. It can detect and identify important sounds in the surrounding environment in real time through the watch, such as sirens, fire alarms, or doorbells.

Song Recognition: With Apple's Shazam app, the watch can automatically detect music playing in the background and accurately display the song name and artist information.

Live Rewind: A conversation rewind feature designed specifically to catch what you missed. For example, when listening to verbal directions or a restaurant menu, if you miss key information, you can simply double-tap the watch's Digital Crown, and a text snippet showing the last 15 seconds of conversation will appear on the watch screen. You can then choose to ignore it (the snippet will permanently disappear) or save it to the standalone Siri app. In addition, below the text, there is an "Ask Siri" interaction option, making it easy to ask follow-up questions about technical terms or unfamiliar words you heard. To protect privacy and ensure awareness, double-tapping the crown triggers a clear notification sound and a watch animation. If not saved manually, these snippets will be automatically deleted after 30 seconds.

Siri Recaps: This feature further extends ambient listening technology to advanced summaries of long conversations. It does not record speaker identities, save raw audio, or generate verbatim transcripts. Instead, it distills core information into a short single sentence plus a bulleted list (such as important dates, deadlines, prices, or payment plans). It is very useful for work meetings, parent-teacher conferences, or personal fitness training sessions where to-dos need to be recorded. The feature automatically pauses when the user is wearing headphones to listen to music or has noise cancellation enabled. Users can receive a recap reminder each morning when unlocking their watch, choose freely to save it to Notes or the Journal app, and if no action is taken, it is automatically deleted after 7 days. Users can also customize the active time window for the feature (for example, running only during work hours and off at night), or turn it off at any time from Control Center.

In terms of privacy and security, Apple emphasizes that the feature is designed to automatically filter sensitive information, including government-issued identifiers, financial and authentication data, specific personal identifiers, and any content promoting self-harm.

Privacy Architecture with Hardware-Software Integration: How On-Device Processing and the Private Cloud Protect Data Security