Spotlight indexing enabled for all volumes after 15.1 update

[gilby101]

I turn off Spotlight indexing for many of my volumes, in particular my non-TM backup, movie stores, virtual machines. But, after the 15.1 update, Spotlight indexed was enabled for ALL volumes. A bit of a surprise.

I have now disabled indexing and deleted the .Spotlight folders.

Something to watch out for.

 

[Polinsky]

You're correct. Didn't notice it. Thanks for posting. Just turned it off on all volumes other than startup volume.

 

[Nermal]

On my first day with 15.1 I ran into two cases where I couldn't eject a flash drive because mds_stores was holding it open. This could explain it!

I have now disabled indexing

Via the UI, or does it require a Terminal command?

 

[gilby101]

Via the UI, or does it require a Terminal command?

I think it needs Terminal. I use:
mdutil -sa to see current status of all volumes (-s status, -a all).
sudo mdutil -i off /Volumes/<volname> to stop indexing of that volume.
sudo mdutil -X /Volumes/<volname> to delete the existing index and its folder /Volumes/<volname>/.Spotlight-V100.

 

[Nermal]

Hmm... mdutil doesn't list my drive and won't stop indexing either ("Error: unknown indexing state"). I wonder whether it's "supposed" to ignore flash drives, but isn't doing it correctly. Yet it's definitely mds_stores holding it open:

% diskutil unmount /dev/disk4s1
Volume UNTITLED on disk4s1 failed to unmount: dissented by PID 320 (/System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Support/mds_stores)

 

[Patrice Brousseau]

Maybe not exactly that but similar: mds_stores was going crazy… even after I thought that Spotlight was completely disabled… but wasn’t!

I had to follow this procedure found on Reddit

https://www.reddit.com/r/MacOSBeta/comments/1dfo2sl/_/l8ua97h

« Removed:
/System/Volumes/Data/.Spotlight-V100

~/Library/Metadata/CoreSpotlight

Set indexing off and deleted the index:

mdutil -a -i off

mdutil -aE »

Then, I enabled Spotlight indexing again… and all went ok, go figure? Maybe it was the deletion of the CoreSpotlight folder that fixed it as it did not reappeared…

 

[gilby101]

Maybe not exactly that but similar: mds_stores was going crazy

mds_stores was, under macOs 15.0.1, writing about 500GB per day. But seems to have calmed down with 15.1. The writing was all to indexes for Spotlight and Core Spotlight (in Photos libraries).

I am trying to work through issues with Spotlight indexing. In my case all related to the mediaanalysisd process which seems to be out of control.

So far:

When I am not actively using my Intel Mac, mediaanalysisd wakes up, consumes CPU (1 core) for an hour, goes into the background for an hour - and repeats.

One issue"

~/Library/Metadata/CoreSpotlight

mediaanalysisd is repeatedly looking for Photos libraries that I deleted long ago. (seen in Console log)

But it not just Core Spotlight. Every hour mediaanalysisd wakes and, presumably, does work (for an hour) on the files (in the file system, not Photos libraries) which it fails to analyse. And wakes again 1 hour later. Reasons I have found are:
1) Images with one dimension is 1 or 2 pixels - mostly lines drawn using PNG files, but also some GIFs and TIF. Mostly ones I can get rid of. All the files shown with mdfind "kMDItemPixelHeight < 3" OR "kMDItemPixelWidth < 3"
2) A few JPEGs which are corrupt (I am investigating further).
3) Four files with colour space Lab. mdfind "kMDItemColorSpace == 'Lab'" finds all four. Is this a general issue with all Lab colour space files or just mine?

I am going to clean out all these images and see if mediananalysisd settles down. But why a) does it repeatedly keep on trying them, and b) why does it need so much CPU for about 50 images?

Very much work in progress for me.

Useful tools:
A) ObjectiveSee FileMonitor with sudo /Applications/Objective-See/FileMonitor.app/Contents/MacOS/FileMonitor -filter mediaanalysisd | grep '"name":"mediaanalysisd"' and saving to a text file. Leave that running when I am away from the Mac - finds all the files that are opened by mediaanalysisd.
B) Console app filtering with by subsystem com.apple.mediaanalysisd. Again go away and let mediaanalysisd do its thing.
C) mdfind commands like those above.

 

[bob_zz123]

I have a habit of putting a .metadata_never_index file in the root of any external drives and disk images I create and this seems to have been honoured still after the 15.1 update:

mdutil -s /Volumes/ExternalSSD
/System/Volumes/Data/Volumes/ExternalSSD:
    Indexing and searching disabled.

 

[Patrice Brousseau]

mds_stores was, under macOs 15.0.1, writing about 500GB per day. But seems to have calmed down with 15.1. The writing was all to indexes for Spotlight and Core Spotlight (in Photos libraries).

I am trying to work through issues with Spotlight indexing. In my case all related to the mediaanalysisd process which seems to be out of control.

So far:

When I am not actively using my Intel Mac, mediaanalysisd wakes up, consumes CPU (1 core) for an hour, goes into the background for an hour - and repeats.

One issue"

mediaanalysisd is repeatedly looking for Photos libraries that I deleted long ago. (seen in Console log)

But it not just Core Spotlight. Every hour mediaanalysisd wakes and, presumably, does work (for an hour) on the files (in the file system, not Photos libraries) which it fails to analyse. And wakes again 1 hour later. Reasons I have found are:
1) Images with one dimension is 1 or 2 pixels - mostly lines drawn using PNG files, but also some GIFs and TIF. Mostly ones I can get rid of. All the files shown with mdfind "kMDItemPixelHeight < 3" OR "kMDItemPixelWidth < 3"
2) A few JPEGs which are corrupt (I am investigating further).
3) Four files with colour space Lab. mdfind "kMDItemColorSpace == 'Lab'" finds all four. Is this a general issue with all Lab colour space files or just mine?

I am going to clean out all these images and see if mediananalysisd settles down. But why a) does it repeatedly keep on trying them, and b) why does it need so much CPU for about 50 images?

Very much work in progress for me.

Useful tools:
A) ObjectiveSee FileMonitor with sudo /Applications/Objective-See/FileMonitor.app/Contents/MacOS/FileMonitor -filter mediaanalysisd | grep '"name":"mediaanalysisd"' and saving to a text file. Leave that running when I am away from the Mac - finds all the files that are opened by mediaanalysisd.
B) Console app filtering with by subsystem com.apple.mediaanalysisd. Again go away and let mediaanalysisd do its thing.
C) mdfind commands like those above.

After the manipulations I have linked, my MacMini 2018 is running ok now and Spotlight is behaving normally after one complete sleep cycle overnight.

Good luck and keep us informed on your progress.

***Oh, CoreSpotlight folder finally rebuilt itself, must be an important part of this.

Edit: also reinstalled the full Sequoia 15.1 package over for peace of mind.

 

[gilby101]

Good luck and keep us informed on your progress.

I believe my Spotlight indexing issues are resolved. For completeness (much of this is in post #7 above):

1) macOS 15.1 update enabling spotlight index on all volumes: Fixed by disabling and erasing index. For the future I have created a .metadata_never_index file in the root those volumes I want to be not indexed - thanks to @bob_zz123

2) Excessive writes by mds_stores: This seems to have been fixed by macOS 15.1. Writes have gone from ~500GB/day down to 1.5 GB/day.

3) mediaanalysisd consuming excessive CPU when I am not using my Mac with 1 hour on and one hour off. Effectively using 1 CPU core flat out for the on hour. Issues and workaround:
3a) Images with zero size or filled with zeros - 6 JPG and 1 TIFF. Workaround by deleting them.
3b) Images (lots) where one or other dimension is less than 3 pixels (e.g. PNGs representing a horizontal line). Workaround by deleting them - I hope this doesn't come back to bite me.
3c) Images using the Lab colour space (only 4 of them from experiments many years ago) - workaround by deleting.
Tools used and some commands are in post #70 above. These files did not cause problems with macOS 14.x and earlier.

4) mediaanalysisd was looking for old (deleted) Photos libraries. This has stopped - I don't know when (maybe also macOS 15.1).

This is all on my Intel 2019 iMac.

mediaanalysisd continues to wake every hour when I am not actively using my Mac. But now finishes very quickly which, I assume, is expected behaviour.

 

[Patrice Brousseau]

Got caught again after swapping an external HDD enclosure for a new one.
Mds stores stuck and macmini refusing to sleep with CPU usage going wild.

I had to redo the mdutil/clear indexes/add remove all volumes again...

We'll see how it goes but Spotlight in 15.1 is buggy!

 

[Speed38]

I think it needs Terminal. I use:
mdutil -sa to see current status of all volumes (-s status, -a all).
sudo mdutil -i off /Volumes/<volname> to stop indexing of that volume.
sudo mdutil -X /Volumes/<volname> to delete the existing index and its folder /Volumes/<volname>/.Spotlight-V100.

When I ran that first command mdutil-sa...Terminal returned this:

Indexing enabled.
/System/Volumes/Data:
Indexing disabled.
/System/Volumes/Preboot:
Indexing enabled.
/Volumes/TimeMachine 1:
Indexing enabled.
/Volumes/TimeMachine 2:
Indexing enabled.

Is this why I am having such problems with Spotlight? Shouldn't indexing be enabled on /System/Volumes/Data?

 

[gilby101]

Is this why I am having such problems with Spotlight? Shouldn't indexing be enabled on /System/Volumes/Data?

Yes! Indexing definitely should be enabled for /System/Volumes/Data

Try sudo mdutil -i on /System/Volumes/Data

 

[Speed38]

When I ran that first command mdutil-sa...Terminal returned this:

Indexing enabled.
/System/Volumes/Data:
Indexing disabled.
/System/Volumes/Preboot:
Indexing enabled.
/Volumes/TimeMachine 1:
Indexing enabled.
/Volumes/TimeMachine 2:
Indexing enabled.

Is this why I am having such problems with Spotlight? Shouldn't indexing be enabled on /System/Volumes/Data?

Here is what finally worked for me...results I posted on another MacRumors forum:

Here is what appears to have fixed it for me:

Restarted in Recovery Mode and ran Disk First Aid on all volumes: all were OK.

• Followed these Apple Instructions:
• Choose Apple menu  > System Settings.
• Open Spotlight settings:
• macOS Sequoia or later: Click Spotlight in the sidebar, then scroll down and click the Search Privacy button on the right.
• Drag the disk or folder that you want to index again to the list of locations that Spotlight is prevented from searching. Or click the add button (+) and select the disk or folder to add
• From the same list of locations, select the disk or folder that you just added. Then click the remove button (–) to remove it from the list so that it can be reindexed.
• Click Done, then quit System Settings. Spotlight will reindex the contents of the disk or folder. This can take some time, depending on the amount of information being indexed.

Waited a minute, invoked Spotlight, searched for several apps in my Apps folder and it failed to find Apps and put them right at the top of the Spotlight hits about 80% of the time nor did Spotlight displayed the word “Indexing” under the results which is usually does when you try and re-index a drive.

So ran Terminal and did this:

Last login: Sun Nov 24 12:31:21 on console
bobby@Roberts-iMac ~ % mdutil -sa
/:
Indexing enabled.
/System/Volumes/Data:
Indexing disabled.
/System/Volumes/Preboot:
Indexing enabled.
/Volumes/TimeMachine 1:
Indexing enabled.
/Volumes/TimeMachine 2:
Indexing enabled.
bobby@Roberts-iMac ~ % sudo mdutil -i on /System/Volumes/Data
Password:
/System/Volumes/Data:
Indexing enabled.
bobby@Roberts-iMac ~ %

 

[Benz63amg]

I'm on Sequoia 15.1.1 and i have ticked OFF all of the boxes on the "Spotlight" settings page in the main Settings app of MacOS. Are you saying that ticking off all of the boxes doesn't fully disable spotlight? i definintley don't want to mess with terminal. are you saying that when i connect an external USB Harddrive to my mac running Sequoia 15.1.1 then Spotlight will start indexing my external drive as well? is there no way to turn that behavior off without terminal?

 

[Speed38]

I'm on Sequoia 15.1.1 and i have ticked OFF all of the boxes on the "Spotlight" settings page in the main Settings app of MacOS. Are you saying that ticking off all of the boxes doesn't fully disable spotlight? i definintley don't want to mess with terminal. are you saying that when i connect an external USB Harddrive to my mac running Sequoia 15.1.1 then Spotlight will start indexing my external drive as well? is there no way to turn that behavior off without terminal?

From whence comes all this "Are you saying"?
I didn't say any of those things enumerated in your post.
Just ask your question which seems to be that you don't want Spotlight to index anything since you say "Are you saying that ticking off all of the boxes doesn't fully disable spotlight?"
If you don't want Spotlight to index anything then, on Settings > Spotlight, go to [Search Privacy] at the bottom and use the + button to add to prevent Spotlight from indexing those locations/drives.

 

[Benz63amg]

From whence comes all this "Are you saying"?
I didn't say any of those things enumerated in your post.
Just ask your question which seems to be that you don't want Spotlight to index anything since you say "Are you saying that ticking off all of the boxes doesn't fully disable spotlight?"
If you don't want Spotlight to index anything then, on Settings > Spotlight, go to [Search Privacy] at the bottom and use the + button to add to prevent Spotlight from indexing those locations/drives.

Thanks for the clarification. My question is this though. By default, When i connect an external harddrive to my M3 Macbook Pro, does MacOS automatically index the external harddrives as well even if i have all the check boxes ticked off under that spotlight settings page? Does that mean i must go into that search privacy settings page and add each of my external harddrives to that list to prevent Spotlight from indexing those locations/drives? when i disconnect the harddrive and reconnect it at a future date, would macos remember to not index that external harddrive or i need to add it to that list each and every time? Thanks

 

[Speed38]

Thanks for the clarification. My question is this though. By default, When i connect an external harddrive to my M3 Macbook Pro, does MacOS automatically index the external harddrives as well even if i have all the check boxes ticked off under that spotlight settings page? Does that mean i must go into that search privacy settings page and add each of my external harddrives to that list to prevent Spotlight from indexing those locations/drives? when i disconnect the harddrive and reconnect it at a future date, would macos remember to not index that external harddrive or i need to add it to that list each and every time? Thanks

I wish I knew the answer to that, but I do not.

While you cannot take answers from sources like Perplexity.ai for gospel, here is what it came up with in answer to your question:

By default, when you connect an external hard drive to your M3 MacBook Pro, macOS automatically indexes the drive using Spotlight, its built-in search utility. This indexing process allows you to search for files on the external drive using Spotlight's search feature. However, there are a few nuances to this behavior:

- **Automatic Indexing**: Generally, macOS will start indexing an external drive when it is connected, provided that the drive is not excluded from indexing in the Spotlight Privacy settings[2].

- **Re-indexing on Reconnection**: If you disconnect and reconnect the drive, macOS may re-index it. This can be particularly noticeable if the drive contains a large amount of data, as the indexing process can take a significant amount of time[3][5].

- **Spotlight Privacy Settings**: You can manage whether a drive is indexed by adjusting the settings in System Preferences under Spotlight > Privacy. If a drive is listed here, it will not be indexed. Removing it from this list will allow indexing to occur[2].

- **Performance Considerations**: The time taken for indexing can vary based on the type of external drive (SSD vs. HDD) and the amount of data stored on it. SSDs generally index faster than HDDs due to their higher data transfer rates[4].

In summary, macOS does automatically index external hard drives by default unless configured otherwise in Spotlight's privacy settings. If you frequently connect and disconnect your external drives, you might notice repeated indexing processes each time the drive is reconnected.

Citations:
[1] https://discussions.apple.com/thread/252536556
[2] https://smallbusiness.chron.com/make-mac-file-names-external-drive-30728.html
[3] https://forums.macrumors.com/threads/spotlight-constantly-indexing-external-ssd.2433501/
[4]

https://www.reddit.com/r/MacOS/comments/f4cmy6

[5] https://forums.macrumors.com/thread...l-drive-after-every-reboot-reconnect.2317358/

 

[Fishrrman]

I "solved" this problem by disabling spotlight entirely.

Been doing that since the very first time spotlight was introduced, quite a few years back.

I read that one can prevent spotlight from indexing a [particular] drive by doing the following:

- create a new file. What I did was to open Text Edit, create a new file, and then just type a short phrase into it (I typed "never index").

- save the file at the root level of the volume you wish NOT TO BE indexed

- give it this name exactly:
".metadata_never_index" (without the quotation marks, note the inclusion of a period before the m)

- it will be saved, but will become an INVISIBLE file (you may have to click through warnings about that, but go ahead and do it).

Once the file is created and saved with that name, it will prevent spotlight from indexing on that volume...

 

[gilby101]

Thanks for the clarification. My question is this though. By default, When i connect an external harddrive to my M3 Macbook Pro, does MacOS automatically index the external harddrives as well even if i have all the check boxes ticked off under that spotlight settings page? Does that mean i must go into that search privacy settings page and add each of my external harddrives to that list to prevent Spotlight from indexing those locations/drives? when i disconnect the harddrive and reconnect it at a future date, would macos remember to not index that external harddrive or i need to add it to that list each and every time? Thanks

@Fishrrman beat me too it. I also now use the .metadata_never_index method to block indexing on many external volumes. Definitely a good way if you have disks only sometimes connected to the Mac.

This is an additional method to add to the post from @Speed38