What are Mac users employing to format large drives?

[MacBook17]

We just took delivery of a WD DC HC550 Ultrastar 16TB HDD - and it is, by far, the largest storage drive we've ever owned. We occasionally need to work with PC machines, so I decided that exFAT and GUID were the best way to go with formatting this new drive, but the problem is that the Disk Utility program refused to format the drive via the fastest setting. Even when I worked with a WD Tech Support rep on this, the ultimate answer was that they would replace the drive. In other words, they didn't have an answer for us.

The machine we're employing for the format task is a mid-2015 MBP15 running via Monterey (12.7.5). Having had no luck with WD tech support, I decided to try the 2-pass setting in Disk Utility and it seems to be working...but it's coming up on 24 hours of run time as I type this. I know that I'm asking about a large storage device, but this still begs some questions:

What are Mac people using to format their large storage devices - in a single pass?

Has anyone been in a similar situation with Disk Utility - and how long did it take to format your large drive?

Cheers for your thoughts on this.

 

[mikes79]

interesting question. If possible, i'd try a more modern macOS or even windows or linux just for the formatting step. I know microsoft recently increased rage maximum partition size for expat, but i would imagine only on the latest OSes. Also you could try partitioning the drive as a workaround. which may or may not help.

 

[Sciuriware]

We just took delivery of a WD DC HC550 Ultrastar 16TB HDD - and it is, by far, the largest storage drive we've ever owned. We occasionally need to work with PC machines, so I decided that exFAT and GUID were the best way to go with formatting this new drive, but the problem is that the Disk Utility program refused to format the drive via the fastest setting. Even when I worked with a WD Tech Support rep on this, the ultimate answer was that they would replace the drive. In other words, they didn't have an answer for us.

The machine we're employing for the format task is a mid-2015 MBP15 running via Monterey (12.7.5). Having had no luck with WD tech support, I decided to try the 2-pass setting in Disk Utility and it seems to be working...but it's coming up on 24 hours of run time as I type this. I know that I'm asking about a large storage device, but this still begs some questions:

What are Mac people using to format their large storage devices - in a single pass?

Has anyone been in a similar situation with Disk Utility - and how long did it take to format your large drive?

Cheers for your thoughts on this.

When you add security passes you are asking for trouble on any system.
You might as well simply format and thereafter fill up the drive with rubbish
to make sure that the empty space no longer contains useful information.
That's quicker and allows you to use it simultaneously.
;JOOP!

 

[bzgnyc2]

We just took delivery of a WD DC HC550 Ultrastar 16TB HDD - and it is, by far, the largest storage drive we've ever owned. We occasionally need to work with PC machines, so I decided that exFAT and GUID were the best way to go with formatting this new drive, but the problem is that the Disk Utility program refused to format the drive via the fastest setting. Even when I worked with a WD Tech Support rep on this, the ultimate answer was that they would replace the drive. In other words, they didn't have an answer for us.

The machine we're employing for the format task is a mid-2015 MBP15 running via Monterey (12.7.5). Having had no luck with WD tech support, I decided to try the 2-pass setting in Disk Utility and it seems to be working...but it's coming up on 24 hours of run time as I type this. I know that I'm asking about a large storage device, but this still begs some questions:

What are Mac people using to format their large storage devices - in a single pass?

Has anyone been in a similar situation with Disk Utility - and how long did it take to format your large drive?

Cheers for your thoughts on this.

GUID/exFAT as you've done is your best bet you want to retain the option to plug an HDD directly into PC computers.

Your 4 basic choices for filesystems on the Mac are: NTFS, APFS, HFS+, and exFAT. Read/write to NTFS on the Mac generally works best using a 3rd party extension. Short of that I would avoid. APFS is not good with HDD and would only use on SSD. Both APFS and HFS+ would require a 3rd party extension to access from Windows which I'm guessing is suboptimal for your environment. Plus people seem to be running into problems with HFS+ on Sonoma now so would avoid for your case.

I don't recall exFAT requiring multiple passes to format a drive but I generally just use it for small partitions on flash drives for compatability between systems so 2 passes could have just flashed by quick enough I didn't notice.

However if you just do the math the base case scenario for any filesystem that needs to do even one pass over a disk that size and speed is ~17 hours (16000 GB / 0.250 GB/sec). If MacOS made you select a 2-pass format and it's still going, I would probably just let it keep running unless you test a solution that requires 0 passes (many filesystems just write key data structures across the disk on creation but don't have to zero every sector of the partition before letting you use the disk).

If you want to experiment to find faster solutions, the one thing I would try is creating an exFAT filesystem yourself from Terminal using "newfs_exfat" on a spare disk on another Mac while you let the above continue uninteruppted. You probably want to specify it use 4096-byte sectors with that drive. If you get it to work with 0 full passes (i.e. just writing a bunch of "superblocks" across the disk but not zero'ing every sector), then it might be worth interrupting the 2-pass format of the above and starting over with what you found with your experiments.

P.S.Also letting the MacOS make 2 passes over the disk might be worth it even with the extra time required just to know your new drive doesn't have any hardware faults depending on what you plan to store on it.

 

[MacBook17]

Excellent post, bzgnyc2! It's been a long time since I received a really helpful post like that!

 

[MacBH928]

what do you mean format? You want to re-write over the disk multiple times to erase what is on it? Or you want to format it to a specific filesystem?

If you want a specific file system like ExFAT it should work. The fact that the WD support said they will replace it shows there might be an issue with the drive itself.

 

[MacBook17]

We just took delivery of a WD DC HC550 Ultrastar 16TB HDD - and it is, by far, the largest storage drive we've ever owned. We occasionally need to work with PC machines, so I decided that exFAT and GUID were the best way to go with formatting this new drive, but the problem is that the Disk Utility program refused to format the drive via the fastest setting. Even when I worked with a WD Tech Support rep on this, the ultimate answer was that they would replace the drive. In other words, they didn't have an answer for us.

Formatting...as in what one does with a "new drive." The "specific file system" was also identified in the OP: exFAT and the GUID partition table. Disk Utility also refuses to format other HDDs, and that brings serious doubt to WD Tech Support's conclusion that the new drive needs to be replaced. Thanks for posting.

 

[chabig]

I decided to try the 2-pass setting in Disk Utility...

This is the part that's confusing us. There is no format that requires 2-passes to complete.

 

[MacBook17]

Sorry, I thought it was clear. I don't have a Windows machine available to me at the moment, so I changed the format method from the fastest setting - which, once again, doesn't work - to the two pass "secure erase" method, which does work. It's been just over 31 hours now, and Disk Utility still reports: Performing two-pass secure erase...

There's so much time into this, at this point, that it's kind of silly to stop the run. I'm simply looking for others who may have been down this road before me. Whether it's called formatting or secure erase - it's still a low-level format, wouldn't you agree?

 

[chabig]

so I changed the format method from the fastest setting - which, once again, doesn't work - to the two pass "secure erase" method, which does work.

To me, a basic format and a secure-erase are separate. But no matter, I agree that you should let it continue. Understand that the computer has to write 16TB to that drive (two times)! before it's complete. That could take many days, or even weeks, depending upon the type of connection.

Here is the link to drive specifications.

https://documents.westerndigital.com/content/dam/doc-library/en_us/assets/public/western-digital/product/data-center-drives/ultrastar-dc-hc500-series/product-manual-ultrastar-dc-hc550-sata-oem-spec.pdf

I don't know how you've connected that drive to your computer, but if you could transfer data at the maximum sustained speed of the drive, 262 MB/s it would require nearly 20 hours to fill the drive once.


I am curious, though, why you wanted to securely erase a new drive? Normally, secure erase is used to obliterate your own data so that it can't be recovered by bad actors. You had no data on that drive. It didn't need a secure erase.

Now, I've made some assumptions. I suspect the erase will complete within 2-3 days. If you were to stop the process now, I think you'd be fine just doing a new format (without secure erase). But if you decide to let it continue, it probably will finish soon.

 

[Sciuriware]

This is the part that's confusing us. There is no format that requires 2-passes to complete.

There is something you all should know:
multi-pass formatting is only useful on rotating platter drives, on SSD it is complete useless.
I attended a conference about security and some guy who worked for secret services or such
showed us that magnetic heads do not always follow the tracks exactly.
He showed pictures of erased disk platters where you could see data outside the tracks
where an expert could still read some erased data.
Multi-pass erasure makes the chance for these remnants a bit smaller but never 100%.
And, as you may know, SSD has no tracks.
;JOOP!

 

[Adora]

Paragon Hard Disk Manager for Mac | Paragon Software

Paragon Software Group delivers solutions for hard disk management, partitioning, drive backup

www.paragon-software.com

It's only for Intel Macs. Might do a better job, but I don't know.

 

[HDFan]

Disk Utility program refused to format the drive via the fastest setting.

I just formatted a 16 TB Seagate Ironwolf Pro with the fastest option on a Silicon Mac. It took ~24 seconds.

I decided to try the 2-pass setting in Disk Utility

Understand that the computer has to write 16TB to that drive (two times)! before it's complete. That could take many days, or even weeks,

A 2 pass erasure is, of course, not needed, but since the fastest option didn't work you had no choice. You'll just have to wait a long time for it to finish.

 

[Adora]

We just took delivery of a WD DC HC550 Ultrastar 16TB HDD - and it is, by far, the largest storage drive we've ever owned. We occasionally need to work with PC machines, so I decided that exFAT and GUID were the best way to go with formatting this new drive, but the problem is that the Disk Utility program refused to format the drive via the fastest setting. Even when I worked with a WD Tech Support rep on this, the ultimate answer was that they would replace the drive. In other words, they didn't have an answer for us.

The machine we're employing for the format task is a mid-2015 MBP15 running via Monterey (12.7.5). Having had no luck with WD tech support, I decided to try the 2-pass setting in Disk Utility and it seems to be working...but it's coming up on 24 hours of run time as I type this. I know that I'm asking about a large storage device, but this still begs some questions:

What are Mac people using to format their large storage devices - in a single pass?

Has anyone been in a similar situation with Disk Utility - and how long did it take to format your large drive?

Cheers for your thoughts on this.

If I have any problem formatting or partitioning a drive, the best solution is always to format the complete drive with JHFS+ and after that there isn't any problem to partition it and use any file system I want for each partition or just format the whole drive with another filesystem.

So I would do that first and after that format to exFAT. Those would be two fast steps instead of this slow one.

 

[Sciuriware]

If I have any problem formatting or partitioning a drive, the best solution is always to format the complete drive with JHFS+ and after that there isn't any problem to partition it and use any file system I want for each partition or just format the whole drive with another filesystem.

So I would do that first and after that format to exFAT. Those would be two fast steps instead of this slow one.

Don't forget that formatting concerns only a fraction of the disk: the volume administration.
Formatting with one of more patterns just writes the entire disk for erasing or testing.
Hard formatting (only for rotating disks) is NEVER done by the customer, because we do not have the software.

In addition to my earlier reply:
official (often ignored) instructions from authorities indicate total destruction of disk drives for the above reasons.
;JOOP!

 

[rebelo]

Hey there. I've faced similar problems with BIG drives and OLD machines and, apparently, it was always a driver issue: old machines/OS don't properly recognize these new drives. It always worked when I plugged the drive on a newer machine and formatted with any filesystem.
Cheers.

 

[MacBook17]

If I have any problem formatting or partitioning a drive, the best solution is always to format the complete drive with JHFS+ and after that there isn't any problem to partition it and use any file system I want for each partition or just format the whole drive with another filesystem.

So I would do that first and after that format to exFAT. Those would be two fast steps instead of this slow one.

Would you please elaborate, Adora? Would you kindly explain how one goes about performing the JHFS+ format you referred to (above)?

 

[MacBook17]

Hey there. I've faced similar problems with BIG drives and OLD machines and, apparently, it was always a driver issue: old machines/OS don't properly recognize these new drives. It always worked when I plugged the drive on a newer machine and formatted with any filesystem.
Cheers.

I'm not understanding you clearly, rebelo. Are you suggesting that we plug the HDD into a newer Mac machine or are you suggesting that the OS we're using may be too old to recognize a 16TB HDD?

 

[bzgnyc2]

Sorry, I thought it was clear. I don't have a Windows machine available to me at the moment, so I changed the format method from the fastest setting - which, once again, doesn't work - to the two pass "secure erase" method, which does work. It's been just over 31 hours now, and Disk Utility still reports: Performing two-pass secure erase...

There's so much time into this, at this point, that it's kind of silly to stop the run. I'm simply looking for others who may have been down this road before me. Whether it's called formatting or secure erase - it's still a low-level format, wouldn't you agree?

I wouldn't call those a low-level format. Low-level formating has historically referred to rewriting the magnetic signal across a rotating disc and usually included things like bad block mapping. I don't believe that has even been possible without drive-specific software since IDE, etc drives came to the scene late 80s. Bad block mapping is all done behind the scenes these days.

Then while colloquial most people say formatting (from the 80s PC-DOS command named format), Apple's Disk Utility and most command-line tools use the terms partitioning, erasing, and/or building/making filesystems. When someone says formatting I usually interpret as the last one (as in 'you need to format the drive as HFS+' -> 'you need to create a new partition and build an HFS+ filesystem on it, etc).

When you partition a drive it also tends to encourage simultaneously build a new filesystem on any newly available partitions, but technically they are two seperate operations. Disk Utility bundles since doing the first without the second isn't what the average user wants.

In any case, truely erasing a drive or partition as in writing all 0s over it isn't usually necessary and most systems usually let you build a new filesystem over old ones. In doing so the directory structures and related are all lost making it difficult to recover any data on that partition. However the data isn't usually overwritten (except in the Secure Erase and similar cases) and it is certainly possible by motiviated people with tools most of don't have access to.

At this point if there's a drive that contained truly sensitive data, I defer to security experts to handle. Sometimes if the data was encrypted (as it usually is if it was sensitive), throwing away all encryption keys is enough. Sometimes they also want to drill holes in or incinerate the drives.

The question in this case is why Disk Utility (the tool I assume you were using) wouldn't successfully "format" that drive without doing a two-pass "secure" erase. Doing the secure erase doesn't hurt anything but shouldn't have been necessary. I'd have to have seen all the settings and messages up until that point to offer any ideas on why you couldn't just do a normal/quick "format".

If you're curious to figure that out you should try playing with a spare, much smaller disk to see what combination of steps and settings requires and doesn't require the secure erase for proceeding. As the WD drive probably came "pre-formatted" for Windows, you might have to pull a drive from that environment back to the Mac to reproduce the circumstances of this case.

 

[MacBook17]

To me, a basic format and a secure-erase are separate. But no matter, I agree that you should let it continue. Understand that the computer has to write 16TB to that drive (two times)! before it's complete. That could take many days, or even weeks, depending upon the type of connection.

Here is the link to drive specifications.

https://documents.westerndigital.com/content/dam/doc-library/en_us/assets/public/western-digital/product/data-center-drives/ultrastar-dc-hc500-series/product-manual-ultrastar-dc-hc550-sata-oem-spec.pdf

I don't know how you've connected that drive to your computer, but if you could transfer data at the maximum sustained speed of the drive, 262 MB/s it would require nearly 20 hours to fill the drive once.

[Image Removed]

The brand new WD Ultrastar 16TB HDD is plugged into a OWC Thunderbolt Drive Dock. As our MBP15 came with (2) TB2 ports, those are the connections we default to whenever possible. Considering that the new HDD is a SATA drive, there's not telling what the actual transfer rate is, but we're employing a Thunderbolt connection to get the best possible connection between the MBP15 and the drive dock.

Based on the bolded bit (above), I'm assuming that we should expect at least 40 hours of run time to achieve a full two-pass write. Am I undertsanding this correctly, chabig?

I am curious, though, why you wanted to securely erase a new drive? Normally, secure erase is used to obliterate your own data so that it can't be recovered by bad actors. You had no data on that drive. It didn't need a secure erase.

Now, I've made some assumptions. I suspect the erase will complete within 2-3 days. If you were to stop the process now, I think you'd be fine just doing a new format (without secure erase). But if you decide to let it continue, it probably will finish soon.

You may have missed it in the OP, but I was forced (more or less) to go with the two-pass erase option. The fastest option simply wouldn't work. Even with HD Tech Support guiding me through it, we couldn't get the fastest erase setting in macOS Disk Utility to run.

I know that there's a subtle difference between the terms Format and Erase, but I've never seen the word "format" used in Disk Utility. Perhaps this has changed in newer macOS versions, but our MBP15 is operating via macOS Monterey (12.7.5), so we're working with what we have. In short, you're correct, we didn't need to perform a two-pass erase, but it was the first erase option that would actually work, so that's what we went with.

We're coming up on 47 hours now and Disk Utility is still reporting the following: "Performing two-pass secure erase...Running."

 

[MacBook17]

I wouldn't call those a low-level format. Low-level formating has historically referred to rewriting the magnetic signal across a rotating disc and usually included things like bad block mapping. I don't believe that has even been possible without drive-specific software since IDE, etc drives came to the scene late 80s. Bad block mapping is all done behind the scenes these days.

Then while colloquial most people say formatting (from the 80s PC-DOS command named format), Apple's Disk Utility and most command-line tools use the terms partitioning, erasing, and/or building/making filesystems. When someone says formatting I usually interpret as the last one (as in 'you need to format the drive as HFS+' -> 'you need to create a new partition and build an HFS+ filesystem on it, etc).

When you partition a drive it also tends to encourage simultaneously build a new filesystem on any newly available partitions, but technically they are two seperate operations. Disk Utility bundles since doing the first without the second isn't what the average user wants.

In any case, truely erasing a drive or partition as in writing all 0s over it isn't usually necessary and most systems usually let you build a new filesystem over old ones. In doing so the directory structures and related are all lost making it difficult to recover any data on that partition. However the data isn't usually overwritten (except in the Secure Erase and similar cases) and it is certainly possible by motiviated people with tools most of don't have access to.

At this point if there's a drive that contained truly sensitive data, I defer to security experts to handle. Sometimes if the data was encrypted (as it usually is if it was sensitive), throwing away all encryption keys is enough. Sometimes they also want to drill holes in or incinerate the drives.

The question in this case is why Disk Utility (the tool I assume you were using) wouldn't successfully "format" that drive without doing a two-pass "secure" erase. Doing the secure erase doesn't hurt anything but shouldn't have been necessary. I'd have to have seen all the settings and messages up until that point to offer any ideas on why you couldn't just do a normal/quick "format".

If you're curious to figure that out you should try playing with a spare, much smaller disk to see what combination of steps and settings requires and doesn't require the secure erase for proceeding. As the WD drive probably came "pre-formatted" for Windows, you might have to pull a drive from that environment back to the Mac to reproduce the circumstances of this case.

This is another very informative post, bzgnyc2. Yes, I completely understand that a secure erase wasn't necessary and, yes, the problem is that Disk Utility wouldn't allow for a fast erase/format. By the way, have you ever seen an actual "format" option in Disk Utility? I can't recall ever seeing the word "format" used in Disk Utility (up to mac OS Monterey). This is why I keep employing the term "erase" in this thread.

I understood the alternate HDD test you described in your first post [thank you again], but we simply don't have another HDD drive dock to work with. Your test is an excellent option, but we simply don't have the ability to try that with another HDD at this point.

 

[Adora]

Would you please elaborate, Adora? Would you kindly explain how one goes about performing the JHFS+ format you referred to (above)?

I think it's called "macOS extended journaled" or something like that in Disk Utility.

Can't see it at the moment because I can't even partition my main drive. I really don't like this APFS and also don''t understand it.

Edit:

 

[Fishrrman]

Just some random thoughts.

It simply doesn't make sense to try to "zero out" (secure erase) a BRAND NEW drive.
There's nothing "on it" to "securely" erase. Nothing there.

I would just cancel out the current attempt -- or just pull the plug and start over.

THIS TIME, DON'T connect "through the hub".
For now, connect the drive DIRECTLY to the Mac's USB port.

Then open disk utility.
Go to the view menu and choose "show all devices".
Look at the list on the left and find the entry (or entries) that represent the WD drive.
Click the line that represents the physical drive itself.
Click the "erase" button.
Enter a drive name and select "Mac OS extended, journaling enabled, GUID partition format".
Let 'er go...

 

[MacBook17]

If you read through the thread, you'll see that I didn't simply choose the two-pass secure erase option. We need the drive as soon as possible and, more to the point, the normal/fast erase option does not work. Once again, the two-pass secure erase option was not a choice - it was done out of necessity.

What does "through the hub" mean?

It's also in the thread, but the machine being used is an MBP15 (mid-2015) running via macOS Monterey (12.7.6). The drive in question - a WD Ultrastar 16TB HC550 7,200 RPM HDD - is plugged into the same OWC Thunderbolt (TB) Dual-Drive Dock that we've used with other HDDs for at least four years now. It has never given us a bit of trouble and it has two TB ports on it...but I digress. The subject HDD is plugged into our OWC Drive Dock and is connected to the MBP15 via an OEM Apple Thunderbolt Cable.

After nearly three full days of run time, the two-pass secure erase has been manually ended. That said, I've tried every combination of formatting available in Disk Utility - via the default/fastest option - and it simply doesn't work. The OS on our MBP15 is working fine, otherwise, so I'm hesitant to try another version of macOS. I suppose it's because we had such a bad experience with macOS Catalina, but a stable OS is hard to let go of these days.

 

[Fishrrman]

I wouldn't usually suggest using anything other than disk utility, but...

Have you tried the WD drive utilities app?

Western Digital Support | Western Digital

Find detailed answers to your support questions for your Western Digital products.

support-en.wd.com

When I said "through the hub", I meant.
DO NOT use a hub or dock.
Connect the drive directly to the Mac, and try erasing it that way.

Do you have A DIFFERENT MAC you could try using?

One more thought:
You are connecting via thunderbolt, correct?
Does the drive also have USB ports on it, by any chance?
If so, have you tried connecting it via USB, and erasing it that way?