Resolved Unsearchable Folder

SpeQ · Mar 21, 2021

That solution might not answer my questions anyway, or more to the point, I might not be able to interpret the information.

I have pasted a few lines of results from the "ls -al /Volumes/disko/" command. My questions are about all the numbers (excluding the dates, of course).

I thought I had a rudimentary understanding after we looked at the file size of the Test 5 folder, but where we saw “total 28440” before, we see either "8" or "256" listed on the line for Test 5 below.

The 699711 number is especially intriguing to me. I would expect that to be a small number.

I have edited the username and one foldername.

Code:

total 1472
drwxrwxrwx@  668 username   staff   21376 Feb  7  2011   foldername

-rw-r--r--@    1 username   staff  699711 Mar  5 18:29 .VolumeIcon.icns

drwxr-xr-x     8 username   staff     256 Mar 20 18:10 test5

SpeQ · Mar 21, 2021

If you right-click on "ls" in Terminal, you are offered the option to open the man page in Terminal. "Services" are also an option, so there's probably a way to automate opening the man page in whatever application. I'm not clear on why you would want to use Preview anyway.

chown33 · Mar 21, 2021

SpeQ said:
That solution might not answer my questions anyway, or more to the point, I might not be able to interpret the information.

I have pasted a few lines of results from the "ls -al /Volumes/disko/" command. My questions are about all the numbers (excluding the dates, of course).

I thought I had a rudimentary understanding after we looked at the file size of the Test 5 folder, but where we saw “total 28440” before, we see either "8" or "256" listed on the line for Test 5 below.

The 699711 number is especially intriguing to me. I would expect that to be a small number.

I have edited the username and one foldername.

Code:

total 1472 drwxrwxrwx@ 668 username staff 21376 Feb 7 2011 foldername -rw-r--r--@ 1 username staff 699711 Mar 5 18:29 .VolumeIcon.icns drwxr-xr-x 8 username staff 256 Mar 20 18:10 test5

The number 8 for "test5" is a hard-link count, not a size. If you add one file in that dir, it should go up by 1. Remove a file and it should go down.

The 256 for "test5" is the size measured in bytes, but that's not too useful for directories. A directory is a list of information on how to find the items within it. It's structured according to the rules for the file-system type: HFS+, APFS, FAT, etc. The size is only of the directory-entries, not of the files within the directory. It's like the difference between an item named on a shopping list ("20 lbs rice") and the actual bag of rice itself.

The actual data in a directory entry varies quite a bit depending on file-system type. It might be a record-number in a B-tree, or a reference to an inode-number, or other esoteric things. I'm not familiar at all with what's in APFS directory entries. All I can really do is infer that a directory entry is fairly small, because 256 (size) divided by 8 (item count) is 32 bytes per entry. Doing the same calculation for "foldername": 21376/668 = 32, so the numbers are consistent for both dirs.

A ".icns" file may contain a bunch of pre-rendered icons at different sizes. As I recall, none of them are in a compressed format, so roughly 700K isn't too surprising to me. I think Preview.app can open ".icns" files, so copy it somewhere, change its name to not start with ".", and open it.

SpeQ · Mar 21, 2021

This is not a problem, I’m just mentioning it because it’s kind of funny. I copied the .VolumeIcon.icns to my desktop and renamed it test.icns, but it still remains hidden (grayed-out). If I do the “command-shift-.” it disappears. It still opens with Preview, and it is a custom drive image I actually made myself, and the drive apparently makes several copies of it at different sizes for display purposes.

But I thought it was humorous that it was still remaining hidden after removing the “.” from the beginning of the name. So I did the “mdls” on it, but if there’s an extended attribute there that’s causing it to be invisible, I’m not seeing it. But I learned something, and it was fun to have a test, even if I failed. Oh wait, maybe it's that "com.apple.icns" is invisible by default?

Code:

_kMDItemDisplayNameWithExtensions      = "test.icns"
kMDItemBitsPerSample                   = 32
kMDItemColorSpace                      = "RGB"
kMDItemContentCreationDate             = 2021-03-05 23:29:31 +0000
kMDItemContentCreationDate_Ranking     = 2021-03-05 00:00:00 +0000
kMDItemContentModificationDate         = 2021-03-05 23:29:31 +0000
kMDItemContentModificationDate_Ranking = 2021-03-05 00:00:00 +0000
kMDItemContentType                     = "com.apple.icns"
kMDItemContentTypeTree                 = (
    "com.apple.icns",
    "public.image",
    "public.data",
    "public.item",
    "public.content"
)
kMDItemDateAdded                       = 2021-03-21 21:22:46 +0000
kMDItemDateAdded_Ranking               = 2021-03-21 00:00:00 +0000
kMDItemDisplayName                     = "test.icns"
kMDItemDocumentIdentifier              = 0
kMDItemFSContentChangeDate             = 2021-03-05 23:29:31 +0000
kMDItemFSCreationDate                  = 2021-03-05 23:29:31 +0000
kMDItemFSCreatorCode                   = "    "
kMDItemFSFinderFlags                   = 16384
kMDItemFSHasCustomIcon                 = (null)
kMDItemFSInvisible                     = 1
kMDItemFSIsExtensionHidden             = 0
kMDItemFSIsStationery                  = (null)
kMDItemFSLabel                         = 0
kMDItemFSName                          = "test.icns"
kMDItemFSNodeCount                     = (null)
kMDItemFSOwnerGroupID                  = 20
kMDItemFSOwnerUserID                   = 501
kMDItemFSSize                          = 699711
kMDItemFSTypeCode                      = "    "
kMDItemHasAlphaChannel                 = 1
kMDItemInterestingDate_Ranking         = 2021-03-05 00:00:00 +0000
kMDItemKind                            = "Apple icon image"
kMDItemLogicalSize                     = 699711
kMDItemOrientation                     = 0
kMDItemPhysicalSize                    = 700416
kMDItemPixelCount                      = 1048576
kMDItemPixelHeight                     = 1024
kMDItemPixelWidth                      = 1024
kMDItemProfileName                     = "sRGB IEC61966-2.1"
kMDItemResolutionHeightDPI             = 144
kMDItemResolutionWidthDPI              = 144

chown33 · Mar 21, 2021

There is a hidden flag that can be applied to files or folders. The flags are only listed with the -O option of ls -l. So: ls -laO then drag the folder containing the icns file to it.

I'm not aware of any 'mdls' data for hiding files, but the hidden flag has been around for a while. Read man chflags for details.

SpeQ · Mar 21, 2021

Ha! I deleted the xattrs and it's now visible! Ha! I learned something!

Now I'm really dangerous. I was much more sloppy than you would have been. It was destined for the trash, so I figured I didn't have much to lose.

chown33 · Mar 21, 2021

SpeQ said:
Code:

cp -afR /Volumes/disko/master /Volumes/disko/test-no-support xattr -d -r "com.apple.metadata:kMDItemSupportFileType" \ /Volumes/disko/test-no-support ls -laeO@ /Volumes/disko/test-no-support

This is the only one that was successful. So stripping the Support File Type attribute makes the files "findable". Or maybe changing that number from 70 to something else might work too(?), but we don't need to go there, unless you just want to for funsies. ?

I'll pass on the funsies.

I assume you can take it from here, regarding how to remove the problematic xattr from every file in your folder of originals. If not, ask again and I can give details.

In any case, I recommend copying the folders of originals to a newly created folder, one that doesn't require you to Show Package to get it to open. Keep the real originals where they are for now, and only work with the copies, for removing xattr's or anything else you were planning. After you're 100% certain that nothing has been lost or borked on the copied originals with xattr's removed, then move the actual originals to safe backup media.

chown33 · Mar 21, 2021

SpeQ said:
Ha! I deleted the xattrs and it's now visible! Ha! I learned something!

Now I'm really dangerous. I was much more sloppy than you would have been. It was destined for the trash, so I figured I didn't have much to lose.

All I can say to that is: Pain can be a wonderful teacher.

SpeQ · Mar 21, 2021

chown33 said:
There is a hidden flag that can be applied to files or folders. The flags are only listed with the -O option of ls -l. So: ls -laO then drag the folder containing the icns file to it.

You lost me there. I don't see anything.

Code:

total 1368
drwxr-xr-x@  3 username  staff  -     96 Mar 21 18:19 .
drwx------@ 15 username  staff  -    480 Mar 21 18:19 ..
-rw-r--r--@  1 username  staff  - 699711 Mar  5 18:29 test2.icns

SpeQ · Mar 21, 2021

chown33 said:
I assume you can take it from here, regarding how to remove the problematic xattr from every file in your folder of originals. If not, ask again and I can give details.

In any case, I recommend copying the folders of originals to a newly created folder, one that doesn't require you to Show Package to get it to open. Keep the real originals where they are for now, and only work with the copies, for removing xattr's or anything else you were planning. After you're 100% certain that nothing has been lost or borked on the copied originals with xattr's removed, then move the actual originals to safe backup media.

Yes, I think I've got it. Actually I don't even plan to use the Originals folder for importing to Photos at this point. I think the Modified files might work better in the long run anyway. I don't want to have to rotate hundreds of pictures again. However, I do plan on making a copy of the Original folder without the xattrs for future use, because I'm sure I'll never remember all this, and I may want it at some point.

I really appreciate you going through all this with me, I've really enjoyed it.

chown33 · Mar 21, 2021

SpeQ said:

You lost me there. I don't see anything.

Code:

total 1368
drwxr-xr-x@  3 username  staff  -     96 Mar 21 18:19 .
drwx------@ 15 username  staff  -    480 Mar 21 18:19 ..
-rw-r--r--@  1 username  staff  - 699711 Mar  5 18:29 test2.icns

You're right, I don't see any flags either. Based on your xattr experiments, it seems there's an xattr for hiding things, along with a hidden flag, and starting a name with dot.

SpeQ said:
Yes, I think I've got it. Actually I don't even plan to use the Originals folder for importing to Photos at this point. I think the Modified files might work better in the long run anyway. I don't want to have to rotate hundreds of pictures again. However, I do plan on making a copy of the Original folder without the xattrs for future use, because I'm sure I'll never remember all this, and I may want it at some point.

I really appreciate you going through all this with me, I've really enjoyed it.

It was an interesting distraction for me from what I should be doing.

Now you can put the "Resolved" prefix on your thread title.

SpeQ · Mar 21, 2021

chown33 said:
It was an interesting distraction for me from what I should be doing.

I know how that goes. Sorry, and You're welcome, simultaneously.

SpeQ · Mar 21, 2021

I hate to post this. I only post it in the furtherance of education.

I copied the iPhoto Library to a new folder on the external drive, and for some reason, the Originals folder is now searchable. We covered this back on comments 25 & 26. I thought we were being pretty meticulous, but something has obviously gotten by me. At this point it appears that simply making a copy of the packaged iPhoto Library into a new folder made the Originals folder searchable. I'm a bit frazzled at this point, but I thought I should go ahead and post this while I'm into it. I'm calling it a night after this.

Here is the mdls for one of the files after copying:

Code:

/P1050620.JPG
_kMDItemDisplayNameWithExtensions      = "P1050620.JPG"
kMDItemAcquisitionMake                 = "Panasonic"
kMDItemAcquisitionModel                = "DMC-FZ20"
kMDItemBitsPerSample                   = 24
kMDItemColorSpace                      = "RGB"
kMDItemContentCreationDate             = 2009-04-03 09:51:18 +0000
kMDItemContentCreationDate_Ranking     = 2009-04-03 00:00:00 +0000
kMDItemContentModificationDate         = 2009-04-03 09:51:18 +0000
kMDItemContentModificationDate_Ranking = 2009-04-03 00:00:00 +0000
kMDItemContentType                     = "public.jpeg"
kMDItemContentTypeTree                 = (
    "public.jpeg",
    "public.image",
    "public.data",
    "public.item",
    "public.content"
)
kMDItemCreator                         = "Ver1.0 "
kMDItemDateAdded                       = 2021-03-21 23:11:11 +0000
kMDItemDateAdded_Ranking               = 2021-03-21 00:00:00 +0000
kMDItemDisplayName                     = "P1050620.JPG"
kMDItemDocumentIdentifier              = 0
kMDItemEXIFVersion                     = "2.2"
kMDItemExposureMode                    = 1
kMDItemExposureProgram                 = 1
kMDItemExposureTimeSeconds             = 0.0015625
kMDItemFlashOnOff                      = 0
kMDItemFNumber                         = 8
kMDItemFocalLength                     = 6
kMDItemFocalLength35mm                 = 36
kMDItemFSContentChangeDate             = 2009-04-03 09:51:18 +0000
kMDItemFSCreationDate                  = 2009-04-03 09:51:18 +0000
kMDItemFSCreatorCode                   = ""
kMDItemFSFinderFlags                   = 1024
kMDItemFSHasCustomIcon                 = (null)
kMDItemFSInvisible                     = 0
kMDItemFSIsExtensionHidden             = 0
kMDItemFSIsStationery                  = (null)
kMDItemFSLabel                         = 0
kMDItemFSName                          = "P1050620.JPG"
kMDItemFSNodeCount                     = (null)
kMDItemFSOwnerGroupID                  = 99
kMDItemFSOwnerUserID                   = 99
kMDItemFSSize                          = 2370962
kMDItemFSTypeCode                      = ""
kMDItemHasAlphaChannel                 = 0
kMDItemInterestingDate_Ranking         = 2009-04-03 00:00:00 +0000
kMDItemISOSpeed                        = 80
kMDItemKind                            = "JPEG image"
kMDItemLogicalSize                     = 2370962
kMDItemMeteringMode                    = 5
kMDItemOrientation                     = 0
kMDItemPhysicalSize                    = 2371584
kMDItemPixelCount                      = 4915200
kMDItemPixelHeight                     = 1920
kMDItemPixelWidth                      = 2560
kMDItemProfileName                     = "sRGB IEC61966-2.1"
kMDItemRedEyeOnOff                     = 0
kMDItemResolutionHeightDPI             = 72
kMDItemResolutionWidthDPI              = 72
kMDItemSupportFileType                 = (
    iPhotoPreservedOriginal
)
kMDItemWhiteBalance                    = 1

chown33 · Mar 22, 2021

The mdls output isn't as useful as listing the xattr's for a file. Furthermore, it should be for a file that's not found in a Finder search.

I don't see a question in your post. You have some unexpected results, but what are you looking for now?

If you want to be fairly sure of having the pictures found in Finder searches, then deleting the xattr named "com.apple.metadata:kMDItemSupportFileType" appears to be reliable. You posted multiple test results showing this earlier in the thread, with no contradicting results: in every case reported, deletion of that xattr made the files appear in Finder search results. In multiple cases, presence of that xattr prevented the files from appearing in Finder search results.

If you want an explanation of why the copied folder contents (with com.apple.metadata:kMDItemSupportFileType xattr's apparently intact) would suddenly become searchable, I don't have one.

I've often seen questions of "Why is X happening?" posted here. That's frequently not the real question they want answered, though, which is "How can I stop X from happening?". Those are two different questions, with different answers. Sometimes one is answerable and the other isn't.

SpeQ · Mar 22, 2021

chown33 said:
I don't see a question in your post. You have some unexpected results, but what are you looking for now?

I'm not asking a question. I was pointing out for anyone who may read this thread that something had gone wrong early on in the thread that might throw everything after that into question. I am pretty much totally confused at this point. What you are saying about the tests we did makes sense.

I have gone back trying to recreate what happened. The (sub)folder that I was using for the tests appears to act differently than others. That folder was the first subfolder within Originals, that was the only reason it was chosen. My current theory is this: iPhoto keeps all originals uploaded to iPhoto in a folder so that the files can't be altered. Maybe if you delete a photo (or in this case, all the photos in that subfolder) within the iPhoto app, the originals are still in the Original folder, but they can't be found.

I have already done my import to Photos of over 10,000 photos, and it wasn't until after I did the import that I went back and tried to recreate the issue, and then came up with this theory. It's not a problem, because if I deleted the photos before, it's not likely I would want them now.

At this point, I don't care much what or why this happened, I just want to go outside

but there may be someone who references this thread that needs to know ...whatever it is I'm trying to relay here. And I thought you might want to know.

SpeQ · Mar 22, 2021

I guess what I’m trying to relay is that all this in unnecessary for someone who wants to import their iPhoto Library to Photos. The files that can’t be found are probably files they put in the iPhoto trash can.

I’m not saying that the methods and procedures for any of the tests are in question. I learned a lot, and I am very appreciative!

Search

Search

Resolved Unsearchable Folder

SpeQ

macrumors regular

SpeQ

macrumors regular

chown33

Moderator

SpeQ

macrumors regular

chown33

Moderator

SpeQ

macrumors regular

chown33

Moderator

chown33

Moderator

SpeQ

macrumors regular

SpeQ

macrumors regular

chown33

Moderator

SpeQ

macrumors regular

SpeQ

macrumors regular

chown33

Moderator

SpeQ

macrumors regular

SpeQ

macrumors regular

Our Staff