iPad File corruption on copy to external storage

[sparksd]

I hear you, but I commented on Files (the goto "official standard file manager" on iOS and iPadOS).

Most users try to use Files, also in connection with ExFAT formatted external and/or movable media of virtually any kind (practically a standard for accessing camera media for at least ten if not fifteen or twenty years).

Apple is not handling ExFAT formatted media well, to be polite.

Since Apple is not necessarily forthright, when it comes to externally connected media, most users will obviously try to use Files. We both know, all reading this thread in fact know, that for certain tasks, there is a fair chance, that transferred data content may be “damaged”, and also the actual content of the external media, may be unreliable after “Apple processing”. Irreparable data loss may be the consequence. If File is used.

If that involves memories from a once in a life time travel - especially if done under the hardships of an epidemic - I'll understand, if a user gets "slightly miffed" (an extremely polite term ;-); especially if that happens as a consequence of an attempt to create a backup in the first place.

Most over the counter (or mail order) retail media also comes ExFAT formatted in order to minimize end user support issues. How many hundred millions of USB sticks and external drives (SSD or HDD) are flung into the world each year?

This “tiny” two billion dollar company, called Apple, is still not able to handle the most wide spread “alternative” file system - ExFAT - for cross platform access in a safe, reliable and trustworthy fashion.

Is that really acceptable?

How are the ordinary users to know, how to avoid putting the safety of their data at risk, when Apple is being very coy - the diplomatic term - in this matter?

I answered the most recent example of a user inadvertently getting “clubbered” by using Files. Probably by no fault of the user.

If iPad Pro users get the idea to try to use Files - an obvious choice for the uninitiated expecting "near notebook performance" (ahem) - they cannot be blamed. We know, that some will get burned and loose data, that cannot be easily replaced. Nowhere does Apple state, that it is a good idea to have a backup of the media, you plan to use with the Files App - depending on the intended use.

Is that fair to the users?

If someone, somewhere gets really annoyed (I was tempted to use a word starting with “p” and accompanied by the word "off" to emphasize the sentiment) and has the connections and wherewithal, there’s a potential law suit - even a class action - waiting to happen.

Regards

I completely agree that it's unacceptable and have been clear on that in my posts on this long thread. I started this thread to raise awareness of the issue and to identify possible alternatives. This is not rocket science and Apple should be able to fix it but they haven't.

 

[Chaparral02]

Do iPhones & Macbooks & Mac’s all suffer from this same problem or is it only apple iPads ?

 

[rui no onna]

Do iPhones & Macbooks & Mac’s all suffer from this same problem or is it only apple iPads ?

iPads and iPhones share the same storage implementation so both would be similarly affected.

 

[UltimoInfierno]

I think I read somewhere that if we use Apple's iCloud, home NAS or Dropbox, then there will be no issue in using Apple's Files?

That’s right, but that doesn’t help you, if you are traveling.

In most hotels, you have only a limited bandwidth. If you’re roaming, let’s say in Vietnam, your US phone bill will quickly accumulate to an average US momthly income - typically long before, you’ve reached the end of your vacation.

You can get local data-SIM’s, but they’re often limited in number per person, and the maximum transfer most often have limits too. At the latest, when you ask for a 25GB refill for the tenths time, there will often be question asked (of you’re getting the refill in you local kiosk instead of online, then owner may be persuaded to show a more lenient approch, depending on the funds coming his way).

The only country, where I‘be experienced good speed in an upscale hotel, the room Ethernet delivered 200 megabit/second for free, but, I guess, that someone hogging the line 24x7 would be in for a serious talk with management.

What remains, is what you carry (or can buy locally). My Cannon camera produces 32 megapixel RAW files in the region 60-65MB on average - each! Or roughly three seconds on a 200 megabit/sec line including overhead. On 4G, it can easily take up to a minute to transfer one file on a good connection.

My Smartphone handles up to 500+ megabit/second for video recordings (typically in the region of 520-540 megabit/sec average, and sustained for minutes on end (have tried 17+ minutes), with short stints up to 580+ megabit/second). One second video will reach around 65 megabyte per second.

Now, that’s roughly, what one Canon RAW image amounts to.

Real easy. One second video requires the same space as one RAW image.

Typically - for once in a life time travels - I end up in the region of (at least) 4-5.000 images after initial removal of clear duds - like lens cap on, my own shoe laces etc. They are to be sorted and weeded out, when I’m home again; I’m not spending valuable travel time on chores like that. The same approach applies to typically around 2-3 hours of video.

Let’s be conservative, and say 4.000 images (260 Gigabyte) and 2 hours of video (7.200 x 65 GB => 468GB) - give and take. A total of 728GB, that can be saved to SSD - especially fast, since we’re talking large files only - at around 200 megabyte per second - or a bit over one hour in total - on and off over a week or two - when performing a backup to SSD via my Smartphone.

I typically copy to internal memory once or twice a day - while on a walkabout - and from there two backups are created each evening in the hotel. I usually create two backups - one in checked luggage, the other in hand luggage. On walkabout, I have the version on my smartphone (500GB plus far slower 400GB on uSDXC, which will hold the most valuable if not all content).

Now, try transferring those 728 GByte of data via cell connections (costly, if even possible at 10 megabit or so). At 200 megabit/sec in a hotel roughly eight hours. It will only be possible in very few, upscale to top end hotels, and in very few countries, regions and cities. In inner old city, central Rome 2019, my hotel delivered around 2-6 megabit/sec in WiFi and roughly the same on my SIM - same rate as local SIM‘s at the same location. Now start backing up to “anything”!

Apples iCloud isn’t much use either under the latter conditions, that are quite normal in most holiday hot spots.

Your own NAS is only of use, if you’re daft enough to allow connections from the internet. That’s like knowingly letting the NSA get free access to the Kremlin communication center. Not done ;-)

Regards

P.S. Skip trying to minimize the problems caused by Apples behavior.

Your wife may have looks, that limit your photos of her to three per vacation, but other people may have other, even larger requirements. Even other “targets”.

None of this will help people, when they don’t even know, that the use of Apples Files app can be detrimental to the safety of your backups in general!

And that‘s where Apple really fails!

 

[hajime]

I guess same dirty Apple tactic. Those who want touch and pen, iPad. Those who want to take nice photos and make callas, iPhone. Those who want to have good file system, Mac. They intentionally limit some functionalities of each line of products hoping that we buy different devices And keep upgrading them.

I guess they intentionally slows down the speed of the Thunderbolt port on M1 iPad Pro (still waiting for confirmation), intentionally put a worse camera on the 2021 model, intentionally made the iPad and iOS file system a mess although they could make it better if they want, intentionally disabled the iPad to use extended screen mode even the M1 is capable of external screen mode, etc. Then later they make some improvements and we pay to upgrade.

 

[sparksd]

I guess same dirty Apple tactic. Those who want touch and pen, iPad. Those who want to take nice photos and make callas, iPhone. Those who want to have good file system, Mac. They intentionally limit some functionalities of each line of products hoping that we buy different devices And keep upgrading them.

I wouldn't go that far. The issue seems limited to the FAT formats - it works well with Apple native formats. And try using an APFS-formatted drive in Windows.

 

[Shirasaki]

I think I read somewhere that if we use Apple's iCloud, home NAS or Dropbox, then there will be no issue in using Apple's Files?

Of course of course. If you use cloud storage Apple file app is half decent, not bad huh. Not what many folks in those thread wants though.
Apple tries so hard to force people forget about files, they still have a long way to go, if that’s what they want people to forget.

 

[Shirasaki]

I wouldn't go that far. The issue seems limited to the FAT formats - it works well with Apple native formats. And try using an APFS-formatted drive in Windows.

That would need a proprietary driver that require admin access to install. Fine for personal device, not so fine for managed devices.

 

[Shirasaki]

I guess same dirty Apple tactic. Those who want touch and pen, iPad. Those who want to take nice photos and make callas, iPhone. Those who want to have good file system, Mac. They intentionally limit some functionalities of each line of products hoping that we buy different devices And keep upgrading them.

I guess they intentionally slows down the speed of the Thunderbolt port on M1 iPad Pro (still waiting for confirmation), intentionally put a worse camera on the 2021 model, intentionally made the iPad and iOS file system a mess although they could make it better if they want, intentionally disabled the iPad to use extended screen mode even the M1 is capable of external screen mode, etc. Then later they make some improvements and we pay to upgrade.

hence we have been playing with hardware subscription model for three decades at this point, only to get this trend accelerated in recent years despite modern devices tend to last quite a while (still not as good as those Commodore 64, Osborne 1 and a sortie of vintage computers).

 

[sparksd]

That would need a proprietary driver that require admin access to install. Fine for personal device, not so fine for managed devices.

Yeah, there are 3rd party apps available - not cheap, either.

 

[UltimoInfierno]

Yeah, there are 3rd party apps available - not cheap, either.

And not really needed, if you don’t use a Mac. Even then, most multi-platform users find other ways to exchange files locally.

 

[Shirasaki]

And not really needed, if you don’t use a Mac. Even then, most multi-platform users find other ways to exchange files locally.

Would you mind sharing other ways to exchange files LOCALLY (absolutely no cloud solution of any kind) without installing third party drivers?

 

[tliu93]

I do the same type of copying - I copy from a photo SD card with RAW images to a T5 or T7 SSD (I have both) formatted exFAT. The storage devices are connected via an Anker hub to my 2021 12.9 Pro. I use FileBrowser Pro to do the copy - it can access both drives in the hub after I add them as locations. I have not seen the same copy failures with it that I have seen with the Files. Is the app free? No, nor would I expect it to be. But it works, making it a good purchase.

Thanks for your info. I have tried both FileBrowser and FE File Explore (both the free version, so it's FileBrowser Go and FE File Explorer free, correct me if I have used the wrong version). Did several tests with both Files and these two apps. It looks like after a few test one of my original files on the SD card was corrupted (not playable at all). I did use microSD so not sure if using an adapter and lock the card would help.

Two other issues found on these two apps:
FE File explorer: not able to 'select all' after filtering the kind of file. GoPro creates many thumbnails/low resolution preview files so I want to copy only the MP4 and WAV files. With FE Explorer I have to select them manually.
FileBrowser: file timestamp cannot be preserved when copying to my T5 SSD. All the files' 'date created' and 'date modified' will be updated to the moment I copied it. while FE File Explorer doesn't have this problem.

Maybe I did something wrong, any recommendations are welcome

 

[tliu93]

Both FileBrowser and FileExplorer have “free versions” (as far as I remember). I have licensed both, but prefer the last, since it allows a kind of “AirDrop” between iOS and Android (both ways). Test them.

The simplest and always working solution is: “Never use Files in iOS or ipadOS for external transfers via USB, if you want to be absolutely sure, that your data survives undamaged”. It really is that simple.

Provocation (but true): Or buy a modern Android smartphone as “travel data manager”, if you want to avoid carrying a PC or Mac notebook along.

My iPads (pro or not) have not been allowed near any external media in earnest the last 12-18 months (Except for testing purposes involving large transfers with File - just to confirm, that file/content damage is still the main trait of using Files).

Apple knows this, but clearly does not make a real effort to remedy the Files behavior, once and for all. Handling ExFAT in a secure way, is not rocket science. Even most really cheap, ten year old point-and-shoot cameras from two-bit outfits work more reliable, than the Files app from the two-billion dollar Apple company. Why is that?

My Samsung Galaxy Note 10+ 500GB (+400 GB internal uSDXC) from 2019 runs rings around any iPhone and most iPads - even my own Pro’s. On the USB-C connection I read up to 275 MByte/s and write up to 250 MByte/s (large video files) directly from an external SanDisk Extreme 500GB (or 1TB) SSD, and using an old and trusted USB-C powered hub I get around 200MB/sec transfers (now SD and uSD cards are also connected, as well as up to several external SSD’s plus the odd ordinary 2TB HDD for extra backup. Naturally while having full output to Full HD - 1080p - HDMI on the hotel TV). With Bluetooth keyboard and mouse - working as a mouse and not a finger replacement - I have a fairly usable desktop replacement (the Samsung scales Word, Excel etc. into “desktop like“ overlapping windows - quite similar to the overlapping windows on Windows or MacOS).

This critter is now on Android 11 with monthly security upgrades; last one this January. Video recording is up to 2268p/HDR10/500+megabit/second h265, when using FilMiC Pro.

I have not experienced damaged media, content or transfer date to/from the Samsung since getting it unpacked. It’s the modern version of Gnarbox 2 (now defunct, it seems), but far more flexible as a media travel tool.

My iPhone 12 Pro only has ProRAW foto format as a benefit, that’s really all (but then, even my many, many years older Panasonic LX100 handles video much better at f1.8 on the comparably “ginormous” near 4/3 format sensor. True bliss for low light videos, when traveling ;-)

If you make sure, that an old Android flagship is still supported, and that the flagship actually supports USB 3.x (some flagships use USB-C but still only support USB 2.0 speeds), you could probably benefit from the better reliability of Android for your use case, compared to iOS/ipadOS.

Regards

Thanks for the suggestions, I have tested them both, both of them have limitations I have to say.

Also I realised these two apps uses a 'connection' mechanism to connect to the external drive, and the UI to select external drive also relies on the underlying iPadOS file system handler I believe. So I am doubting whether these apps can handle the exFat file system differently from the original Files app, without corrupting my files.

I also realised FileBrowser copies files more slowly, with some kind of 'examine and calculate' step so probably more reliable, while FE FileExplorer did faster. But FileBrowser couldn't preserve my file timestamp. I am now still doubting whether these app will work fine

 

[tliu93]

I think I read somewhere that if we use Apple's iCloud, home NAS or Dropbox, then there will be no issue in using Apple's Files?

Yes I do have a custom build server acting ad my NAS as well, running samba server there for file sharing between all my devices. I have tried to connect to my samba server with both Files on my iPad as well as iPhone, they both work like a charm, even more stable than using Finder on my Mac.

 

[sparksd]

Thanks for your info. I have tried both FileBrowser and FE File Explore (both the free version, so it's FileBrowser Go and FE File Explorer free, correct me if I have used the wrong version). Did several tests with both Files and these two apps. It looks like after a few test one of my original files on the SD card was corrupted (not playable at all). I did use microSD so not sure if using an adapter and lock the card would help.

Two other issues found on these two apps:
FE File explorer: not able to 'select all' after filtering the kind of file. GoPro creates many thumbnails/low resolution preview files so I want to copy only the MP4 and WAV files. With FE Explorer I have to select them manually.
FileBrowser: file timestamp cannot be preserved when copying to my T5 SSD. All the files' 'date created' and 'date modified' will be updated to the moment I copied it. while FE File Explorer doesn't have this problem.

Maybe I did something wrong, any recommendations are welcome

The Files app does not preserve the dates either so this appears to be an iPadOS "feature" (even with Windows copying, the Date Created timestamp is not preserved; it is set to the time of the copy (Date Modified is preserved)). I have never run into the source file corruption issue with FB Pro.

 

[tliu93]

The Files app does not preserve the dates either so this appears to be an iPadOS "feature" (even with Windows copying, the Date Created timestamp is not preserved; it is set to the time of the copy (Date Modified is preserved)). I have never run into the source file corruption issue with FB Pro.

Yes indeed I know that, I get that Date Created will be updated. The problem is, from my experience, the 'Date Modified' filed will also be updated by FileBrowser. And for photos it will be fine since I can retrieve the original creation date from EXIF, but for videos this will make my life much more difficult.

 

[sparksd]

Yes indeed I know that, I get that Date Created will be updated. The problem is, from my experience, the 'Date Modified' filed will also be updated by FileBrowser. And for photos it will be fine since I can retrieve the original creation date from EXIF, but for videos this will make my life much more difficult.

As I said, this appears to be an iPadOS feature, not unique to FB.

 

[Slartibart]

Yes indeed I know that, I get that Date Created will be updated. The problem is, from my experience, the 'Date Modified' filed will also be updated by FileBrowser.

”modified” will not change if you transfer the files via the ”Share”-function in FileBrowser oder FE FileExplorer.

If your videos are on Apple’s Photos app on your iPhone/iPad you share them via FB or FEFE to the folder whereever - NAS, a shared folder on a Windows machine, external SSD, etc. (left/above is the dialogue coming up when you share via FEFE, right/below Filebrowser Pro. Yes, in the pic there are JPEGs, but it’s the same for mp4&co. 🤓)

 

[tliu93]

As I said, this appears to be an iPadOS feature, not unique to FB.

As I mentioned, this also happens to FileBrowser, but FEFE can preserve 'Date modified'.

 

[tliu93]

As I said, this appears to be an iPadOS feature, not unique to FB.

And I can confirm that for FEFE the 'Date created' field is also preserved when copying. I'm not sure it will be better for paid version, but at least the free version (FileBrowser Go) I tried doesn't preserve any timestamp. See screenshots, the first two videos are copied by FEFE and the last to are copied by FB. They are videos shot on the same day (13 Jan, 2022).

I am just wondering whether FB has the feature to preserve timestamp or not.

 

[tliu93]

”modified” will not change if you transfer the files via the ”Share”-function in FileBrowser oder FE FileExplorer.

If your videos are on Apple’s Photos app on your iPhone/iPad you share them via FB or FEFE to the folder whereever - NAS, a shared folder on a Windows machine, external SSD, etc. (left/above is the dialogue coming up when you share via FEFE, right/below Filebrowser Pro. Yes, in the pic there are JPEGs, but it’s the same for mp4&co. 🤓)

View attachment 1945602 View attachment 1945603

Thank you for the suggestion. But this option is not so viable for me, as I'm copying files from SD card to external SSD.

 

[Need_Advice]

I decided to give the iPad another try. So just to sum it up… for backing up raw files from an SD card directly to an SSD (without saving them on the iPad), FileBrowser Professional is the way to go, right?

 

[Slartibart]

I decided to give the iPad another try. So just to sum it up… for backing up raw files from an SD card directly to an SSD (without saving them on the iPad), FileBrowser Professional is the way to go, right?

as well as FE FileExplorer. Each of both has additional advantages: e.g. FileBrowser Professional allows for easy renaming of multiple files at once, FE FileExplorer allows copying in the background (it creates a queue for all transfers which you can “hide”).
FileBrowser Pro operates tab-oriented, that is you open a tab for each “location” and the copy between them, FE FileExplorer‘s GUI is similar to Apple’s Files: you select the files and choose e.g. to transfer them selecting the destination from the copy-dialogue.

Considering the prices I bought and use both. 🤓

EDIT: both exceed Apple’s Files in reliability as well as functionality - both support various protocols to connect to external drives incl. SMB 1. Both allow to use the iPad as a file server for other devices. And again, while using Apple’s framework they work much more reliable the Apple’s Files.

One more thing: at the moment there seems to be no functionality provided in iPadOS 13/14/15.3.1 to get an info on the free space of an external device. You can get an info of the occupied space for e.g. an SSD or NAS-drive, as a result you can initiate a file transfer with a total size which will not fit on the destination device/location and you end up with an error in the transfer log. As said, this is something originating from the underlying iPadOS, not FBPro or FEFE.

Maybe one last thing: due to the transfer-queue which one can run in the background as well as a progress indicator for transfers which includes an rate estimator for kB/Mb per second I find myself at the moment using FEFE more often. But I really like the tab-oriented approach to file operations FBPro offers as well as the rename-multiple-files function. First world problems. 😎 YMMV.

 

[Need_Advice]

Thanks! And no issues so far with the same method I mentioned in my previous comment?