iOS 18's Rumored AI Features for Siri, Spotlight, and More Revealed in New Report

[smithrh]

I've mentioned this elsewhere, but already in current versions of MacOS and iOS Apple is indexing and analyzing and combining data locally on devices like crazy, it's a huge increase from what plain-old Spotlight was doing.

BiomeAgent, biomesync. photoanalysisd, mediaanalysisd, IntelligencePlatformd, knowledgeAgent and more have been working away in the background to help populate devices with personal data even before the OSs arrive.

 

[MilaM]

How does this summarization work? Say you pull up a New York Times story. How does the AI decide what are the most important points to summarize?

It does not really "decide". It is very hard if not impossible to reason about the exact steps an LLM takes to create the summary. But apparently it works quite well (most of the time).

 

[MilaM]

On DuckDuckGo you can play around with one of the OpenAI models for free, if anyone is interested. It's not the latest one, but seems to be good enough to get an idea how useful "AI" might be. No sign-up required.

DuckDuckGo at DuckDuckGo

duckduckgo.com

I'm really curious how Apple is going to integrate LLMs into iOS. I still have a hard time seeing a business case for the general public, but hey, I also thought crypto is useless, so ...

 

[jeremz1ne]

Well, these features are rumored to be announced next month.
There will be no iPhone 16 Pro by next month.
So… no.

until they hit us with an early iphone 16 event

 

[MrSegundus]

Sounds like it won’t be very useful or not as useful as the Samsung AI features. Very weak.

 

[AppliedMicro]

Text summarisation has been available and built into Mac OS X, running on-device since (at least) the days of Mac OS X 10.4 "Tiger", 15 years ago:

So I guess the news is: Something similar may soon come to an iOS device near you as an "Artificial Intelligence" feature.

In other recent news: Artificially intelligent Siri fails to tell the time when you ask it.

 

[klasma]

We’ll see if it will be able to accurately summarize MacRumors forum threads.

 

[maxoakland]

Wait so their big feature ideas are… summarizing web pages and responding to text messages for you?

These aren’t exciting features and frankly responding to messages for me is an off-putting idea. I don’t want to be less connected to the people in my life or have an AI bot in between our communication

 

[purplerainpurplerain]

I can quickly summarize a thread myself without consuming 1 megawatt of energy and monthly subscription to ClosedAI.

And so can you.

You just did it. 😛

 

[TheNewLou]

I'm really curious how Apple is going to integrate LLMs into iOS. I still have a hard time seeing a business case for the general public, but hey, I also thought crypto is useless, so ...

Criminals & scammers? Checks out.

 

[TheNewLou]

Wait so their big feature ideas are… summarizing web pages and responding to text messages for you?

These aren’t exciting features and frankly responding to messages for me is an off-putting idea. I don’t want to be less connected to the people in my life or have an AI bot in between our communication

Even autocomplete is weird this way. It'll often choose a synonym to the word I was going to use, but it often doesn't sound like "my voice".

It does have its uses, though, as autocorrect just warned me about my typo "chose" when I meant "choose" in the sentence above, lol. It's better for grammar than word choice, I'd say (although it messes up grammar a lot).

 

[jamesjingyi]

I think the biggest highlights of Apple's AI push will be the on-device processing. Basically every single AI feature you find on other devices sends it to a server to be processed, which could make sense in the short term, but long term poses a few issues:

1. Reliance on an internet connection — It might not sound like a big problem, especially if it's integrated into something like a phone where you are 'always connected', but it will become more and more relevant the bigger the files or features get. If something like a photo or video needs to be processed, you need to factor in the time to upload, process, and re-download the output. Spotty or even just slower internet connections will just mean that a lot of the features 'feel' slow, or just fail altogether.
2. Reliance on services — You're reliant on the other party continuing to 'allow' the use of that service, and that that other party exists. Currently ‘circle to search’, the Google AI feature, is free and isn’t capped, but eventually it might be restricted or even charged as a monthly fee (see the Pixel’s ‘unlimited free photo backups’). Chat GPT currently allows free use of 3.5, but they could decide that actually everyone has to pay at some point, and if certain features rely on a connection like this, you don’t want to be left out in the cold, especially if it was a selling point of the phone. All of the ‘AI’ features we’ve seen so far just send stuff to a server to be processed, so actually it’s not reliant on hardware at all, all software locks (see the Pixel 8 Pro AI features vs the base Pixel 8). That to me is not as impressive as doing it all locally, since it's just basically 'pre-paying' for access to their servers.
3. Privacy — I think it just sounds nicer to know that all the processing is happening locally. I know you can 'opt-out' of the use of your data for training or research, but really, it's still my data that's travelling back and forth. Personally I was kinda horrified when I saw all the recordings that Google had from my Google Home (enough for me to ditch it completely). Even without saying the wake word it was recording and storing my data, and that was just all there being stored in their servers, potentially being used for training (maybe for voice, but also for any of their other stuff, like ads?) Yeah there might be some optional diagnostic data that can be sent back, but with most of the current AI offerings, you're kinda are stuck because you have to send stuff to a server to be processed.
   
   Look at what Apple can do with your photo library, all with local processing, without it being sent to their servers. Google sends that all off-device to be processed. Yes I back up everything in iCloud, but I know that Apple's servers don't have to look at my photos to give me those neat search features. Google has to process them to allow search.
4. Responsiveness — Having less things in the pipeline will mean that stuff happens much quicker. Seeing the reviews of the Humane AI Pin and the R1 show that even a direct connection takes time to process. Personally, seeing the difference between my Apple Watch Series 6 vs Ultra 2 with on-device voice processing, it's completely night and day. Instead of waiting for my voice to be sent to a server over the internet, processed, interpreted, sent back, and then actioned, most of that can just happen right on the device, and it's so much faster.
5. Features — There's quite a few things that AI has not been rolled out to, not because it can't do it, but rather because it requires access to things that most people wouldn't just 'hand over' to something that would be sending things back and forth on the internet. I think that's a key part of why Humane and Rabbit made their own hardware. We might not see it with this initial drop, but I think local implementation will allow things that just would be either 'too expensive' to send and get back from servers, would be considered 'not secure enough' like paying for things, or would require too many 'back and forths'. An example of this would be Rewind, a Mac App that records everything you do and lets you search it, and ask questions. That all runs locally, both for privacy, but also because having a constant stream of your screen(s) to a server to be processed would just be insanely expensive.

I think Apple's been planning for the future with the emphasis on their NPUs in the recent A and M-series chips. We've seen a lot of stuff recently from AMD and Intel with NPUs, but since most stuff still happens in the cloud, they are just underutilised. Notable exceptions to this of course include some of Adobe's AI stuff, but in terms of OS-level integrations, most stuff goes off-device (Co-Pilot is all internet based afaik, though there are plans for more on-device stuff in W12)

 

[contacos]

Well, these features are rumored to be announced next month.
There will be no iPhone 16 Pro by next month.
So… no.

You think this will come out before iOS 18 in September? Doubt it. It already takes Apple months to release a .X release and 17.5 is probably the last release before the 18 beta to WWDC in June

 

[jz0309]

more "genAI" "features, rumors, hype ... what problem is genai actually trying to solve for the average consumer? haven't seen anything yet ... sure, some will see some benefit but again, the average consumer???

 

[JPack]

So will this require an iPhone 16 pro?

No doubt about it. Will likely require iPhone 16 series with A17+ for all the juicy features.

Developers can start working on it immediately by using iPhone 15 Pro. But there's always a bunch of features Apple will reserve for the new iPhones.

 

[macabrumorsab]

I think the biggest highlights of Apple's AI push will be the on-device processing. Basically every single AI feature you find on other devices sends it to a server to be processed,

I'll try with facts instead of thinking. On my Z Fold 5 the following is processed on-device:

• Interpreter feature
• Live translation during phone call (I am not joking)
• Transcription of voice recordings
• Remving of objects, blur, reflections and shadows
• Parts of the AI-related features of Samsung notes, Samsung Keyboad, chats in messaging apps

There might be more, that I am not aware of.
There is also a system setting to limit the AI to on-device-only feature set.
Just because Apple claims they would do something different, it does not mean this is true.

 

[Rogifan]

Those parts that align with its bias 😀

They talk about ultra processed food... now we will be getting ultra processed news.

At work Outlook scolds me telling me I should try being more concise. Now Safari will be decided for me what’s most important in a story. I don’t want any of it.

 

[Rogifan]

No doubt about it. Will likely require iPhone 16 series with A17+ for all the juicy features.

Developers can start working on it immediately by using iPhone 15 Pro. But there's always a bunch of features Apple will reserve for the new iPhones.

So does Apple talk about any of that at WWDC? It would be hard to say ‘coming this fall…but only for new hardware that hasn’t been released yet’.

 

[hagar]

It will be very underwhelming if Apple simply sprinkles some basic AI features around the OS. What I want is a personal assistant. A local Chat-GTP style smarts that has access to all my personal data, that knows me and my life and will be PRO-ACTIVE in making me run my life.

 

[purplerainpurplerain]

It will be very underwhelming if Apple simply sprinkles some basic AI features around the OS. What I want is a personal assistant. A local Chat-GTP style smarts that has access to all my personal data, that knows me and my life and will be PRO-ACTIVE in making me run for my life.

Fixed it for you.

What happens if you live in a country where some three letter agency demands all that data be handed over to them because you went to protest against some dictator or oligarch?

Hope you don’t think people like Altman who were groomed by Thiel are your friends.

 

[masotime]

Text summarisation has been available and built into Mac OS X, running on-device since (at least) the days of Mac OS X 10.4 "Tiger", 15 years ago:

I watched the video - all it is really doing is picking sentences, not actually summarizing the content. It's very different from what LLMs can do now, which is to actually understand context and rewrite sentences which are more succinct.

 

[maxoakland]

It will be very underwhelming if Apple simply sprinkles some basic AI features around the OS. What I want is a personal assistant. A local Chat-GTP style smarts that has access to all my personal data, that knows me and my life and will be PRO-ACTIVE in making me run my life.

ChatGPT calculates the most statistically likely response to your input. It’s autocorrect

You’re making a very common mistake thinking it knows anything or could make any decisions. That’s not how it works

I don’t think it’s possible right now to make any kind of AI that would actually be good at those things

And maybe that’s OK. I don’t think the privacy and personal implications of letting an application run my life are worth the slight convenience. All software has bugs. That’s especially bad if it’s doing life things for you

 

[Aslak3]

Summarising the text on a web page… what can go wrong?

 

[CarAnalogy]

Wait so their big feature ideas are… summarizing web pages and responding to text messages for you?

These aren’t exciting features and frankly responding to messages for me is an off-putting idea. I don’t want to be less connected to the people in my life or have an AI bot in between our communication

Also how long are these text messages that people need a summary? How hard is it to just read?

I wouldn’t trust Siri’s summary anyway.

 

[return2sendai]

I still haven’t personally used AI for anything and don’t know how I would even start considering I keep my devices pretty organized and it’s easy for me to search things myself.

Most of this Ai stuff is really about us having to learn how to do what we already do in a different way. Put simply, novelty. Which soon wears off when we go back to doing what we used to do in the same way we used to do it before the Ai stuff came along.