I wonder if I can turn off iCloud Photos and then force the phone to "re-learn" / delete all cache !?
I'm not sure why you'd need to force iCloud Photos to "relearn" - essentially each image stores a last-viewed date, and that date is used to determine priority for removal. It's very similar to the way RAM is managed on Mac - data/code remains RAM-resident based on a combination of its last-accessed date/time and the system's need to move new data/code into RAM.
If you turn off iCloud Photos you'll encounter several options: "Remove from iPhone," "Download Photos & Videos," and (fortunately) "Cancel." So yes, you could do this to "delete all cache." Assuming you were to "Remove from iPhone" and then activate iCloud Photos again, you'll end up re-downloading the full library of thumbnails and few, if any, full-quality images. In my experience the full-quality images won't download until you try to view them (basically, this is the same thing that happens after someone erases and restores a device, or sets up a new one).
It could be a way to minimize the size of the on-device library, but because iCloud upload/download activity requires separate storage space for temporary caching (it's part of the "Other" in Settings > General > iPhone Storage) I've seen situations where the library gets stuck in limbo - nothing uploading or downloading due to an absence of available device storage for the temporary cache.
My feeling is, unless it's clear that something is clearly malfunctioning trust Optimize Storage to manage the on-device "cache." As you saw in my previous post, my on-device storage is about 3-4% of the size of the cloud-based library, and it's likely half of that 3-4% is thumbnails, so maybe I'd recover 1-2 GB by doing the whole "Remove from iPhone" thing, and that savings would evaporate as soon as I started viewing old images and taking new images. Doesn't seem worth it.