I am just a normal computer user. However, my understanding is when you replace the folder by another folder with the same name. What you do is just make the file system that can't locate the old files anymore, but those files still physically exist on your HDD. Lots of data recovery software can easily locate those files (by scan through the entire HDD) and recovery them (basically just register those data as a files in the OS again).
I another word, in the software point of view, the file is deleted. However, in the hardware point of view, the data still there. Therefore, as long as there is one software can revalidate those data. That deleted file will magically re-appear again in the file system (software side).
What secure delete is actually a write action, not delete action. It delete a file on the software side, but actually over write all the data with 0 on the hardware side. Therefore, it takes much longer to finish the "delete" action. However, since the data was basically destroyed in both hardware and software side, it's much much harder to recover the data.
SSD may be a little bit different, because once the files mark as deleted by the OS, TRIM will tell the SSD controller that those data is no longer required, therefore, GC will physically destroy those data and free up the cell for future write action. This is how SSD keep the write performance. So, assume you leave the computer switch on, and let the SSD idle for some time. Those "quick deleted files" may actually completely destroyed on both software and hardware world.