iPad File corruption on copy to external storage

[sparksd]

Well, yes, but these things are essentially all workarounds... I think it's more than strange on Apple's part that a device which is marketed as a laptop replacement is simply unusable for such an essential basic thing like copying files to/from storage devices with a widely used file system.

I would much prefer not having to purchase more and more special devices as a workaround, but being able to use normal cross-compatible devices like with any other laptop/tablet.

As I mentioned, I'm now using exFAT external storage extensively without a problem, but on iOS 14.

 

[Aldwych]

Yes, thank you! I will try to update to iPadOS 14. I'll keep you in the loop. Cheers!

 

[sparksd]

Yes, thank you! I will try to update to iPadOS 14. I'll keep you in the loop. Cheers!

Please do. If there is a new, different manifestation of this issue we need to run it to ground and I'd be happy to help with any testing on my end.

 

[Slartibart]

Of course! So, the Samsung T5 is formatted with exFAT out of the box, I haven't reformatted it since. ....

I would usually copy large amounts of RAW photos (about 25 MB each) or video files (ranging from a few MB to >4 GB) in either direction (iPad –> external storage and vice versa). File corruption irregularly happens in both directions. The source files are never corrupted (this only happened with the iOS Files app), corruption only occurs with the destination files. ....

iPP 11” 2020, iPadOS 14.3, FileBrowser for Business, Samsung T5, Uni USB-C hub - similar use scenario, only: no problems with exFAT, actually no more file system inconsistencies, damaged RAWs/JPEGs, copy problems as long as I mount and dismount the external medium in Filebrowser BIZ (and use it to copy the files).

4. Open two instances of FileBrowser side-by-side.

why do you do that? You can simply open additional tabs - this allows even direct copying between 2 external media without copying it to the iPad (like on a real computer ????).

....
8. Turn off iPad.
9. Unplug external storage device.
10. Turn on iPad.

well, the important bit is to mount (and dismount before removal) the external medium in FileBrowser. Then no shutdown/restart is required. See attached screen shot, connected/mounted directories or media have this yellow circled connectors-icon, non-connected/unmounted directories/media show 3 points (I circled some in red). click this to mount/unmount your T-5 or whatever.

EDIT: for the described configuration and as long as I do all copy operations from and to external directly connected media within FileBrowser, all the problems described here (and which I suffered previously with Apple’s Files) are gone - no damaged photos or videos independent of number or size.

The problem FileBrowser does currently not/can’t solve - you can not develop RAWs directly on a mounted SD-card. Well, actually you can, Pixelmator Photo or Affinity Photo allow that technically. But the apps using Apple’s Files to access the SD card and after ending up a few times with an unreadable SD card I can not recommend doing that. Made a backup before...with FileBrowser. ??

 

[sparksd]

iPP 11” 2020, iPadOS 14.3, FileBrowser for Business, Samsung T5, Uni USB-C hub - similar use scenario, only: no problems with exFAT, actually no more file system inconsistencies, damaged RAWs/JPEGs, copy problems as long as I mount and dismount the external medium in Filebrowser BIZ (and use it to copy the files).

why do you do that? You can simply open additional tabs - this allows even direct copying between 2 external media without copying it to the iPad (like on a real computer ????).

well, the important bit is to mount (and dismount before removal) the external medium in FileBrowser. Then no shutdown/restart is required. See attached screen shot, connected/mounted directories or media have this yellow circled connectors-icon, non-connected/unmounted directories/media show 3 points. E.g. click this to mount/unmount your T-5.

I take that "dismount" one step further - I put the iPad to sleep (not shut down) and then remove the storage. My trust in iOS with external storage is still a bit frayed ...

 

[Aldwych]

Hello @Slartibart, thanks for your insight!

why do you do that? You can simply open additional tabs - this allows even direct copying between 2 external media without copying it to the iPad (like on a real computer ????).

Quite simply because I prefer to see the folders side-by-side as opposed to switching to and fro between tabs. FileBrowser explicitly supports this functionality, so why not use it?

well, the important bit is to mount (and dismount before removal) the external medium in FileBrowser. Then no shutdown/restart is required. See attached screen shot, connected/mounted directories or media have this yellow circled connectors-icon, non-connected/unmounted directories/media show 3 points. E.g. click this to mount/unmount your T-5.

Hm, are you quite sure that this actually mounts/unmounts the device from the system? I'm under the impression that volumes get mounted automatically when plugged in (as can be seen in the Files app) and the icons you showed just link the mounted volume to a shortcut inside FileBrowser.

 

[sparksd]

Hello @Slartibart, thanks for your insight!



Quite simply because I prefer to see the folders side-by-side as opposed to switching to and fro between tabs. FileBrowser explicitly supports this functionality, so why not use it?


Hm, are you quite sure that this actually mounts/unmounts the device from the system? I'm under the impression that volumes get mounted automatically when plugged in (as can be seen in the Files app) and the icons you showed just link the mounted volume to a shortcut inside FileBrowser.

I talked to the FileBrowser guys about the dismount - there really isn't one in iOS. They said to watch the drive light to make sure that there aren't any transfers going on before disconnecting. I just took the further step of putting the iPad to sleep. And yes, given the way the connection is made in FileBrowser, I also believe it is just a link to the volume that Files has established.

 

[Slartibart]

hem

I talked to the FileBrowser guys about the dismount - there really isn't one in iOS. They said to watch the drive light to make sure that there aren't any transfers going on before disconnecting. I just took the further step of putting the iPad to sleep. And yes, given the way the connection is made in FileBrowser, I also believe it is just a link to the volume that Files has established.

You are of course right - there is actually no real unmount/eject in iPadOS. Apple’s Files seems to assign a fixed buffer to a mounted external medium. A few small files are copied quite fast and you have (usually) no problem.
But if you copy several files/big chunk of data something happens - often the Files app just shuts down or you end up with file system inconsistencies on you external drive or partial damaged files.
There is something quite wrong with Apple’s implementation - I use “buffer” here descriptive, based on my observations like first few files are copied fast and then everything slows done to a crawl (well, it feels like that, because Apple sadly still does not provide reliable user feedback on copy operations from/to external media) or that read flags of files suddenly become damaged, bits are missing,... there are of course other possible explanations.

For every non-beta iPadOS version Apple releases I usually do my personal small “let’s copy a few hundred pictures from and to my iPad”-test with Apple’s Files. And so far (since iPadOS 13) each time I give them feedback that it is not reliable working. I encourage everyone to do so too.

These problems never occur when copying to a cable connected NAS (the Uni-hub use allow that) - not really surprising because it’s a different I/O/protocol.

You mentioned the RAVPower wireless card reader - I can only second your recommendation, great device, works reliable...even with Apple’s Files. But WLAN transfers are no fun with many/larger files. ? I do bigger photo projects in intervals every year, and when doing so I usually end up with at least a few hundred RAWs per day so directly copying from/working on the SD cards is much faster. Quick sort, copy and develop (some of) them on my iPad somewhere in e.g. forestal northern Scandinavia is possible thanks to FileBrowser.

 

[Aldwych]

There is something quite wrong with Apple’s implementation

Absolutely – I think this can be declared the motto of this thread

 

[sparksd]

hem

You are of course right - there is actually no real unmount/eject in iPadOS. Apple’s Files seems to assign a fixed buffer to a mounted external medium. A few small files are copied quite fast and you have (usually) no problem.
But if you copy several files/big chunk of data something happens - often the Files app just shuts down or you end up with file system inconsistencies on you external drive or partial damaged files.
There is something quite wrong with Apple’s implementation - I use “buffer” here descriptive, based on my observations like first few files are copied fast and then everything slows done to a crawl (well, it feels like that, because Apple sadly still does not provide reliable user feedback on copy operations from/to external media) or that read flags of files suddenly become damaged, bits are missing,... there are of course other possible explanations.

For every non-beta iPadOS version Apple releases I usually do my personal small “let’s copy a few hundred pictures from and to my iPad”-test with Apple’s Files. And so far (since iPadOS 13) each time I give them feedback that it is not reliable working. I encourage everyone to do so too.

These problems never occur when copying to a cable connected NAS (the Uni-hub use allow that) - not really surprising because it’s a different I/O/protocol.

You mentioned the RAVPower wireless card reader - I can only second your recommendation, great device, works reliable...even with Apple’s Files. But WLAN transfers are no fun with many/larger files. ? I do bigger photo projects in intervals every year, and when doing so I usually end up with at least a few hundred RAWs per day so directly copying from/working on the SD cards is much faster. Quick sort, copy and develop (some of) them on my iPad somewhere in e.g. forestal northern Scandinavia is possible thanks to FileBrowser.

When I communicated with the FileBrowser guys, they were at a loss as to why their implementation worked and Files had issues as they both are using the same internal iOS interfaces.

I quite agree on your last paragraph. I used to take my laptop on travel just to work with RAW images - doing some processing, sorting, culling and backing up. But now I just use my SSD and a USB-C hub. I still do my primary processing at home on the desktop - just more efficient work flow, has a larger screen, NAS access and real DAM tools. I still take the FileHub along - I've had to use its Ethernet connection at times and also to create a local network to get around hotel restrictions on number of device connections. It's been a handy little device.

 

[Slartibart]

When I communicated with the FileBrowser guys, they were at a loss as to why their implementation worked and Files had issues as they both are using the same internal iOS interfaces.

they implemented some form of checking wether for an individual file the copy is started and finished, the minimalistic feedback Apple gives is just the standard activity indicator spinNing while an unquantifiable task is executed.

Written that, it might be something multitasking related...

What I find so surprising is that basically everyone I know who connects an external storage medium via USB directly to an iPad runs into this problem sooner or later. And it is something which is easily to reproduce - I mean if one wants to push it, just connect an SD card and start developing RAWs but then don’t save them on the iPad but export JPEGs, PNGs or TIFFs directly back onto the card. At best after a few files you end up with damaged ones, at worst the card becomes unreadable for iPadOS (newly formatting in the camera makes it at least usable again).

 

[Aldwych]

What I find so surprising is that basically everyone I know who connects an external storage medium via USB directly to an iPad runs into this problem sooner or later. And it is something which is easily to reproduce - I mean if one wants to push it, just connect an SD card and start developing RAWs but then don’t save them on the iPad but export JPEGs, PNGs or TIFFs directly back onto the card. At best after a few files you end up with damaged ones, at worst the card becomes unreadable for iPadOS (newly formatting in the camera makes it at least usable again).

Exactly!! How Apple still hasn't fixed this and there's no real uproar about it is beyond me.

 

[Slartibart]

to paraphrase Mr. Balmer: Developer, Developer, Developer, Send Feedback, Send Feedback, Send Feedback!

 

[Aldwych]

They said to watch the drive light to make sure that there aren't any transfers going on before disconnecting.

I'm currently experimenting again, keeping this in mind. Interestingly, the drive light never stops blinking as long as it is attached to the iPad and the iPad is turned on!

 

[Aldwych]

I only connected and disconnected the SSD when the iPad was turned off completely. Now I reattached it to my PC – and immediately got the "Do you want to scan and fix Samsung_T5 (N:\)?" window. Yes, I'm still on iPadOS 13, but this nevertheless shows the annoying erratic behaviour that I've been describing in this thread. I'm currently preparing for the iPadOS 14 update (backups etc.).

 

[Aldwych]

Ok, I have finally updated the iPad to iPadOS 14.4.

But: Same thing as in my previous message. Turned iPad off, waited generously, disconnected drive, plugged it into my PC, "scan and fix" window appeared.

 

[sparksd]

Ok, I have finally updated the iPad to iPadOS 14.4.

But: Same thing as in my previous message. Turned iPad off, waited generously, disconnected drive, plugged it into my PC, "scan and fix" window appeared.

Yeah, it's frustrating that it keeps happening. The file system flag that indicates proper unmounting isn’t getting set. When you repair the drive, it indicates no problems were found. If you skip the repair and run chkdsk on the drive it will indicate that there are no problems. In this particular case, I usually ignore the warning and use the drive.

 

[Aldwych]

Thanks for this perspective! However:

When you repair the drive, it indicates no problems were found.

Not always! It has found errors several times and thus there is now a folder on the SSD called "FOUND.000" with "FILEnnnn.CHK" files inside. It really is frustrating, as I was hoping that updating to iPadOS 14 would solve this behaviour.

 

[sparksd]

Thanks for this perspective! However:


Not always! It has found errors several times and thus there is now a folder on the SSD called "FOUND.000" with "FILEnnnn.CHK" files inside. It really is frustrating, as I was hoping that updating to iPadOS 14 would solve this behaviour.

After using the Files app?

 

[Aldwych]

After using the Files app?

After doing nothing at all. I.e.: Connecting drive, turning on iPad, doing normal tasks without using the SSD (SSD drive light is blinking the whole time), turning off iPad, waiting until everything is 100% off, disconnecting drive, connecting drive to PC.

 

[sparksd]

After doing nothing at all. I.e.: Connecting drive, turning on iPad, doing normal tasks without using the SSD (SSD drive light is blinking the whole time), turning off iPad, waiting until everything is 100% off, disconnecting drive, connecting drive to PC.

Weird. I see the repair issue but not any corruption or error problems (at least since I stopped using Files) with my SSD that I regularly use back and forth with my iPP and Win10 machine.

 

[ThunderSkunk]

Interesting. I wonder how widespread this problem is in Apples ecosystem. I have my MBP’s ssd split into 4 partitions: 1 apfs for os x, 1 ntfs for boot camp, & 2 exFAT partitions to hold the engineering & architecture projects both OS’s need to be able to work on, reflecting our networks file directory structure and updating both directions in a synchronization scheme.

If OS X and Apples own internal SSD’s have the same problem and have corrupted the files in my projects on the exFAT partitions, I’ll have been syncing those corrupt files back down to the server for months, and across months of backups, overwriting good data with bad. The next time we end up reactivating an affected project, we’ll find out if my insistence on using Apple computers cost us a days work or a court case.

Great. Can’t wait.

 

[sparksd]

Interesting. I wonder how widespread this problem is in Apples ecosystem. I have my MBP’s ssd split into 4 partitions: 1 apfs for os x, 1 ntfs for boot camp, & 2 exfat partitions to hold the engineering & architecture projects both OS’s need to be able to work on, reflecting our networks file directory structure and updating both directions in a synchronization scheme.

If OS X and Apples own internal SSD’s have the same problem and have corrupted the files in my projects, I’ll have been syncing those corrupt files back down to the server for months, and across months of backups, overwriting good data with bad. The next time we end up reactivating an affected project, we’ll find out if my insistence on using Apple computers cost us a days work or a court case.

Great. Can’t wait.

The corruption issues I have run into have always been with storage formatted exFAT.

 

[Slartibart]

Great. Can’t wait.

You’ll have to, because OSX Finder does not have the described problems as long as you dismount (eject) any connected device (wether it is directly connected or networked).

We are talking iOS/iPadOS here. And just for the protocol: on iPadOS 13/14 there seem to be no problems accessing in any form data on a network drive/partition with whatever supported file system... the same hardware gives you the previously described problems when connected directly to your iPad/iPhone, but if accessed via LAN/WLAN everything is fine... this is the infuriating bit...

 

[Ray2]

I thought I’d add my experience with the Files app. I’m using a Gen 6 Lightning iPad, the Lightning SD card reader and the Files app. I’ve used iOS 13 and 14.

I travel 4 to 6 months each year. Probably take 20,000 images per year of which the vast majority go from sd card to iPad using the Files app. More specifically, using Files, I copy images from the card directly into the Adobe CC folder. I do not use Lightroom for this.

In my 10's of 1,000's of images, over 100's of discreet transfers, using the Files app, on old and obsolete hardware, with iOS 13 and 14 I’ve never seen a corrupt raw or jpeg file.

I haven’t read the entire thread but I sure don’t have any issues with the Files app.