So, the results.
Installation on APFS Encrypted went fine. After the reboot, before you can apply the post install patches, you need to unlock the volume first in terminal via diskutil apfs unlockVolume /dev/diskXsY. After that, the patcher can see the volume and apply the patches. It did it without complaining. I checked afterwards and it worked properly: everything needed on the EFI partition was there, same for the apfsbootuuid folder in the root of my High Sierra disk.
But as probably could be expected, startup.nsh was unforunately not able to boot it, because it can't find the boot file, and it can't because the volume at that point is locked/encrypted.
@dosdude1 Do you think a solution here is possible? Can you unlock the volume from the EFI shell? Ask the user for the passphrase? Or is the EFI shell too limited for that.
That was all done by formatting the disk as APFS Encrypted first and
then installing High Sierra on it.
Now I'm going to try installing on a normal, not encrypted APFS, and then turning on the FileVault 2. We'll see what happens.
This workaround is a better solution to the APFS boot problem, though, as it avoids creating unnecessary helper boot partitions for every bootable APFS High Sierra volume you have. So we have working FaceTime/iMessage
and a better APFS boot method. We should stick to it

.