Spotlight indexing enabled for all volumes after 15.1 update

[gilby101]

I turn off Spotlight indexing for many of my volumes, in particular my non-TM backup, movie stores, virtual machines. But, after the 15.1 update, Spotlight indexed was enabled for ALL volumes. A bit of a surprise.

I have now disabled indexing and deleted the .Spotlight folders.

Something to watch out for.

 

[Polinsky]

You're correct. Didn't notice it. Thanks for posting. Just turned it off on all volumes other than startup volume.

 

[Nermal]

On my first day with 15.1 I ran into two cases where I couldn't eject a flash drive because mds_stores was holding it open. This could explain it!

I have now disabled indexing

Via the UI, or does it require a Terminal command?

 

[gilby101]

Via the UI, or does it require a Terminal command?

I think it needs Terminal. I use:
mdutil -sa to see current status of all volumes (-s status, -a all).
sudo mdutil -i off /Volumes/<volname> to stop indexing of that volume.
sudo mdutil -X /Volumes/<volname> to delete the existing index and its folder /Volumes/<volname>/.Spotlight-V100.

 

[Nermal]

Hmm... mdutil doesn't list my drive and won't stop indexing either ("Error: unknown indexing state"). I wonder whether it's "supposed" to ignore flash drives, but isn't doing it correctly. Yet it's definitely mds_stores holding it open:

% diskutil unmount /dev/disk4s1
Volume UNTITLED on disk4s1 failed to unmount: dissented by PID 320 (/System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Support/mds_stores)

 

[Patrice Brousseau]

Maybe not exactly that but similar: mds_stores was going crazy… even after I thought that Spotlight was completely disabled… but wasn’t!

I had to follow this procedure found on Reddit https://www.reddit.com/r/MacOSBeta/comments/1dfo2sl/_/l8ua97h
« Removed:
/System/Volumes/Data/.Spotlight-V100

~/Library/Metadata/CoreSpotlight

Set indexing off and deleted the index:

mdutil -a -i off

mdutil -aE »

Then, I enabled Spotlight indexing again… and all went ok, go figure? Maybe it was the deletion of the CoreSpotlight folder that fixed it as it did not reappeared…

 

[gilby101]

Maybe not exactly that but similar: mds_stores was going crazy

mds_stores was, under macOs 15.0.1, writing about 500GB per day. But seems to have calmed down with 15.1. The writing was all to indexes for Spotlight and Core Spotlight (in Photos libraries).

I am trying to work through issues with Spotlight indexing. In my case all related to the mediaanalysisd process which seems to be out of control.

So far:

When I am not actively using my Intel Mac, mediaanalysisd wakes up, consumes CPU (1 core) for an hour, goes into the background for an hour - and repeats.

One issue"

~/Library/Metadata/CoreSpotlight

mediaanalysisd is repeatedly looking for Photos libraries that I deleted long ago. (seen in Console log)

But it not just Core Spotlight. Every hour mediaanalysisd wakes and, presumably, does work (for an hour) on the files (in the file system, not Photos libraries) which it fails to analyse. And wakes again 1 hour later. Reasons I have found are:
1) Images with one dimension is 1 or 2 pixels - mostly lines drawn using PNG files, but also some GIFs and TIF. Mostly ones I can get rid of. All the files shown with mdfind "kMDItemPixelHeight < 3" OR "kMDItemPixelWidth < 3"
2) A few JPEGs which are corrupt (I am investigating further).
3) Four files with colour space Lab. mdfind "kMDItemColorSpace == 'Lab'" finds all four. Is this a general issue with all Lab colour space files or just mine?

I am going to clean out all these images and see if mediananalysisd settles down. But why a) does it repeatedly keep on trying them, and b) why does it need so much CPU for about 50 images?

Very much work in progress for me.

Useful tools:
A) ObjectiveSee FileMonitor with sudo /Applications/Objective-See/FileMonitor.app/Contents/MacOS/FileMonitor -filter mediaanalysisd | grep '"name":"mediaanalysisd"' and saving to a text file. Leave that running when I am away from the Mac - finds all the files that are opened by mediaanalysisd.
B) Console app filtering with by subsystem com.apple.mediaanalysisd. Again go away and let mediaanalysisd do its thing.
C) mdfind commands like those above.

 

[bob_zz123]

I have a habit of putting a .metadata_never_index file in the root of any external drives and disk images I create and this seems to have been honoured still after the 15.1 update:

mdutil -s /Volumes/ExternalSSD
/System/Volumes/Data/Volumes/ExternalSSD:
    Indexing and searching disabled.

 

[Patrice Brousseau]

mds_stores was, under macOs 15.0.1, writing about 500GB per day. But seems to have calmed down with 15.1. The writing was all to indexes for Spotlight and Core Spotlight (in Photos libraries).

I am trying to work through issues with Spotlight indexing. In my case all related to the mediaanalysisd process which seems to be out of control.

So far:

When I am not actively using my Intel Mac, mediaanalysisd wakes up, consumes CPU (1 core) for an hour, goes into the background for an hour - and repeats.

One issue"

mediaanalysisd is repeatedly looking for Photos libraries that I deleted long ago. (seen in Console log)

But it not just Core Spotlight. Every hour mediaanalysisd wakes and, presumably, does work (for an hour) on the files (in the file system, not Photos libraries) which it fails to analyse. And wakes again 1 hour later. Reasons I have found are:
1) Images with one dimension is 1 or 2 pixels - mostly lines drawn using PNG files, but also some GIFs and TIF. Mostly ones I can get rid of. All the files shown with mdfind "kMDItemPixelHeight < 3" OR "kMDItemPixelWidth < 3"
2) A few JPEGs which are corrupt (I am investigating further).
3) Four files with colour space Lab. mdfind "kMDItemColorSpace == 'Lab'" finds all four. Is this a general issue with all Lab colour space files or just mine?

I am going to clean out all these images and see if mediananalysisd settles down. But why a) does it repeatedly keep on trying them, and b) why does it need so much CPU for about 50 images?

Very much work in progress for me.

Useful tools:
A) ObjectiveSee FileMonitor with sudo /Applications/Objective-See/FileMonitor.app/Contents/MacOS/FileMonitor -filter mediaanalysisd | grep '"name":"mediaanalysisd"' and saving to a text file. Leave that running when I am away from the Mac - finds all the files that are opened by mediaanalysisd.
B) Console app filtering with by subsystem com.apple.mediaanalysisd. Again go away and let mediaanalysisd do its thing.
C) mdfind commands like those above.

After the manipulations I have linked, my MacMini 2018 is running ok now and Spotlight is behaving normally after one complete sleep cycle overnight.

Good luck and keep us informed on your progress.

***Oh, CoreSpotlight folder finally rebuilt itself, must be an important part of this.

Edit: also reinstalled the full Sequoia 15.1 package over for peace of mind.

 

[gilby101]

Good luck and keep us informed on your progress.

I believe my Spotlight indexing issues are resolved. For completeness (much of this is in post #7 above):

1) macOS 15.1 update enabling spotlight index on all volumes: Fixed by disabling and erasing index. For the future I have created a .metadata_never_index file in the root those volumes I want to be not indexed - thanks to @bob_zz123

2) Excessive writes by mds_stores: This seems to have been fixed by macOS 15.1. Writes have gone from ~500GB/day down to 1.5 GB/day.

3) mediaanalysisd consuming excessive CPU when I am not using my Mac with 1 hour on and one hour off. Effectively using 1 CPU core flat out for the on hour. Issues and workaround:
3a) Images with zero size or filled with zeros - 6 JPG and 1 TIFF. Workaround by deleting them.
3b) Images (lots) where one or other dimension is less than 3 pixels (e.g. PNGs representing a horizontal line). Workaround by deleting them - I hope this doesn't come back to bite me.
3c) Images using the Lab colour space (only 4 of them from experiments many years ago) - workaround by deleting.
Tools used and some commands are in post #70 above. These files did not cause problems with macOS 14.x and earlier.

4) mediaanalysisd was looking for old (deleted) Photos libraries. This has stopped - I don't know when (maybe also macOS 15.1).

This is all on my Intel 2019 iMac.

mediaanalysisd continues to wake every hour when I am not actively using my Mac. But now finishes very quickly which, I assume, is expected behaviour.

 

[Patrice Brousseau]

Got caught again after swapping an external HDD enclosure for a new one.
Mds stores stuck and macmini refusing to sleep with CPU usage going wild.

I had to redo the mdutil/clear indexes/add remove all volumes again...

We'll see how it goes but Spotlight in 15.1 is buggy!